Data reading method, client, server and storage system

Abstract

The invention relates to the technical field of databases, and provides a data reading method, a client, a server and a storage system. The data reading method is applied to a client. The method comprises the following steps: receiving a first operation; according to the first operation, cyclically executing the following steps S1 to S3: S1, determining an idle connection from the N KV database connections; s2, a data reading request is sent to the server through the idle connection, the data reading request comprises a first command, a data identifier of the target data, a reading vernier and a reading step length, and the first command is used for instructing the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the first reading vernier and the reading step length; s3, receiving a reading result returned by the server, wherein the reading result comprises a data segment corresponding to the target data; and when a first preset condition is met, stopping circulation, and splicing all the read data segments according to a reading sequence to obtain target data.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data reading method, client, server, and storage system. Background Art

[0002] Redis is a widely used key-value database (KV database) that uses key-value pairs as storage objects and supports storing values ​​of various data types. In Redis, each key corresponds to a unique value, and clients can retrieve the corresponding value through the key. It is often used in scenarios such as caching.

[0003] Strings are a common data type. When stored in a KV database, they can be used directly as values ​​or as the smallest element in a collection. However, when reading data from a KV database, some strings can have large values, which can easily cause the KV database or client to be blocked during the reading process, thereby reducing the efficiency of the KV database and / or client in processing requests. Summary of the Invention

[0004] The present application provides a data reading method, client, server and storage system, which solves the problem of low efficiency of KV database and / or client processing requests in the prior art.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a data reading method is provided, which is applied to a client, and the client reads data from a KV database through N KV database connections established between the client and the server, where N≥1. The method comprises: receiving a first operation, the first operation being used to instruct the client to read target data from the KV database; according to the first operation, cyclically executing the following steps S1 to S3: S1, determining an idle connection from the N KV database connections; S2, sending a data reading message to the server through the idle connection, the data reading message including a first command, a data identifier of the target data, a first reading cursor and a reading step, the first command being used to instruct the server to read the target data according to the data identifier, the reading cursor and the reading step. Step length reads the data segment corresponding to the data identifier from the KV database; S3, receives the reading result returned by the server, the reading result includes the data segment corresponding to the target data, the reading cursor of the target data in the next cycle and the indication information whether the target data has been read completely; when the first preset condition is met, the loop is stopped, and all the read data segments are spliced ​​in the reading order to obtain the target data; wherein, in each loop, the data identifier of the target data is the same; the reading step length in the kth loop is determined by the client according to the reading result received in the k-1th loop, and in the kth loop, the first reading cursor is updated to the second reading cursor, k≥2.

[0007] Optionally, the read step length in the data read message may be 0 or the data read message does not include a read step length. When the data read message does not include a read step length or the read step length in the data read message is 0, the server executes the first command according to the data read message to obtain the data segment corresponding to the data identifier using a preset read step length, where the preset read step length may be a first threshold, such as 250 KB.

[0008] In the method provided in this embodiment, the client can obtain the target data by cyclically reading received data segments. That is, in each cycle, after obtaining an idle connection, the client only reads data of a certain read step length. When it is determined that the target data has been read, the read data segments are spliced ​​to obtain the target data. This method can solve the problem that due to the large amount of data read in a single time, the KV database connection is occupied for a long time, resulting in other requests being unable to obtain an idle connection, thereby reducing the execution efficiency of the client and KV database threads and affecting the normal reading of other data.

[0009] In some embodiments, the read result further includes a second version number of the target data recorded in the KV database.

[0010] In some embodiments, the data read request includes a first version number, and the method further includes: in the kth cycle, before sending the data read request to the server through the idle connection, updating the first version number to a second version number.

[0011] In some embodiments, the method further includes: determining a third version number, where the third version number is the second version number of the target data recorded in the KV database included in the reading result of the k-1th cycle.

[0012] In this embodiment, when the server returns the read result to the client, it returns the data segment corresponding to the target data, the second read cursor of the target data in the next cycle, and the indication information of whether the target data has been fully read. At the same time, it returns the second version number of the target data recorded in the KV database to the client, so that the client updates the first version number to the second version number before sending a data read message to the server through an idle connection in the kth cycle, or after the kth cycle ends; at the same time, it determines the third version number. In this way, the client can determine whether the target data has been modified based on whether the version numbers included in the two consecutive read results received are consistent; in addition, the server can also determine whether the target data has been modified when it determines that the version number in the received data read message is inconsistent with the version number recorded in the KV database.

[0013] In some embodiments, the first preset condition includes: the second version number is consistent with the third version number, wherein the second version number is consistent with the third version number to indicate that the target data has not changed, and the first indication information sent by the server is received, and the first indication information is used to indicate that the target data has been read completely.

[0014] In this embodiment, when the second version number and the third version number of the client are consistent, it indicates that the target data has not changed during the data reading process; at the same time, the client receives the first indication information sent by the server, indicating that the target data has been read completely. At this time, the client stops the loop and splices all the read data segments in the reading order to obtain the target data.

[0015] In some embodiments, the method further includes: stopping the loop and deleting all read data segments when a second preset condition is met.

[0016] In some embodiments, the second preset condition includes: the second version number is inconsistent with the third version number, wherein the inconsistency between the second version number and the third version number is used to indicate that the target data has changed; or, an error message sent by the server is received, and the error message is generated by the server in the kth cycle when it is determined that the version number in the received data reading message is inconsistent with the version number recorded in the KV database, and the error message is used to indicate that the target data has changed.

[0017] Among them, when the second version number of the client is inconsistent with the third version number, or the client receives an error message sent by the server, indicating that the target data in the KV database has changed, the client stops the loop and deletes all the data segments read, thereby avoiding the problem of dirty reading or missed reading.

[0018] In some embodiments, the process of determining the read step length in the kth cycle includes: determining according to the response time of the read result received in the k-1th cycle; or determining according to the number of read results received.

[0019] In this embodiment, the client determines the reading step length in the kth cycle based on the reading result received in the k-1th cycle, so that the reading step length in each cycle can be adjusted during the target data reading process to adapt to the actual reading process, thereby improving data reading efficiency.

[0020] In some embodiments, the read step size is less than or equal to a first threshold.

[0021] In this embodiment, when determining the reading step length in the kth cycle, the client controls each reading step length to be less than or equal to the first threshold value to avoid the problem of blocking the KV database due to the value of the reading step length being set too large.

[0022] It should be noted that after the client obtains an idle connection, the reading step length of the target data in the first data reading message sent to the server through the idle connection can be a preset step length, that is, when k=1, the reading step length is the preset step length.

[0023] A second aspect provides a data reading method, which is applied to a server deployed with a KV database, and is applied to the server to read data from the KV database by executing a database operation command. The method comprises: receiving a data reading message sent by a client, the data reading message including a first command, a data identifier of target data, a read cursor, and a read step size, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database based on the data identifier, the read cursor, and the read step size; executing the first command according to the data reading message to obtain the data segment corresponding to the data identifier; and sending a read result to the client when a third preset condition is met, the read result including the data segment of the target data.

[0024] In this embodiment, after receiving the data reading message, the server executes the first command in the data reading message, reads the data segment corresponding to the data identifier from the KV database using the data identifier, read cursor and read step size, and returns the reading result of the data segment including the target data to the client when the third preset condition is met.

[0025] The read result also includes a second read cursor of the target data in the next cycle executed by the client and indication information of whether all the target data has been read.

[0026] In some embodiments, the data read message also includes a version number of the target data.

[0027] The version number can be 0 or not.

[0028] In some embodiments, the third preset condition includes: the version number is 0; or, the version number is not 0, and the version number is consistent with the version number of the target data recorded in the KV database.

[0029] In this embodiment, when the version number in the data reading message is 0, the server does not need to verify the version number, determines that the third preset condition is met, and sends the reading result to the client; when the version number in the data reading message is not 0, the version number needs to be verified.

[0030] In some embodiments, the method further includes: generating an error message when the third preset condition is not met, the error message being used to indicate that the target data has changed; and sending the error message to the client.

[0031] In this embodiment, when the version number in the data reading message is not 0, and the version number in the data reading message is inconsistent with the version number of the target data recorded in the KV database, it indicates that the target data has changed, and it is determined that the third preset condition is not met. An error message is generated and sent to the client to instruct the client to stop the target data reading process, thereby avoiding the problem of dirty reading or missed reading by the client.

[0032] In some embodiments, according to the data reading message, a first command is executed to obtain the data segment corresponding to the data identifier, including: when the reading step is 0, according to the data reading message, the first command is executed to obtain the data segment corresponding to the data identifier with a preset reading step; when the reading step is not 0, according to the data reading message, the first command is executed to obtain the data segment corresponding to the data identifier with the reading step.

[0033] Optionally, if the data reading message does not include a reading step length, the server executes the first command according to the data reading message to obtain a data segment corresponding to the data identifier with a preset reading step length.

[0034] According to a third aspect, a client is provided, which reads data from a KV database through N KV database connections established with a server, where N≥1. The client includes: a receiving module for receiving a first operation, where the first operation is used to instruct the client to read target data from the KV database; an operation module for cyclically executing the following steps S1 to S3 according to the first operation: S1, determining an idle connection from the N KV database connections; S2, sending a data reading message to the server through the idle connection, where the data reading message includes a first command, a data identifier of the target data, a reading cursor, and a reading step size, and the first command is used to instruct the server to read the target data according to the data identifier, the first reading cursor, and the reading step size. Read the data segment corresponding to the data identifier from the KV database; S3, receive the reading result returned by the server, the reading result includes the data segment corresponding to the target data, the second reading cursor of the target data in the next cycle, and indication information of whether all the target data have been read; the processing module is used to stop the loop when the first preset condition is met, and splice all the read data segments in the reading order to obtain the target data; wherein, in each loop, the data identifier of the target data is the same; the reading step size in the kth loop is determined by the client based on the reading result received in the k-1th loop, and in the kth loop, the first reading cursor is updated to the second reading cursor, k≥2.

[0035] The fourth aspect provides a server, which is deployed with a KV database, and includes: a receiving module for receiving a data reading message sent by a client, the data reading message including a first command, a data identifier of the target data, a reading cursor and a reading step, wherein the first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the reading cursor and the reading step; an execution module for executing the first command according to the data reading message to obtain the data segment corresponding to the data identifier; and a sending module for sending the reading result to the client.

[0036] In a fifth aspect, a storage system is provided, comprising the client shown in the third aspect and the server shown in the fourth aspect.

[0037] It can be understood that the beneficial effects of the third to fifth aspects mentioned above can be found in the relevant descriptions of the first to second aspects mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the structure of a storage system provided in one embodiment of the present application;

[0039] Figure 2A A schematic flowchart of different service threads in the same client reading data from a KV database provided in an embodiment of the present application;

[0040] Figure 2B A schematic flowchart of different clients reading data from a KV database provided in an embodiment of the present application;

[0041] Figure 3A A schematic diagram of a client writing data into a database according to an embodiment of the present application;

[0042] Figure 3B A schematic diagram of a client reading data from a database according to an embodiment of the present application;

[0043] Figure 4 A schematic diagram comparing the amount of data written by different key values ​​in the KV database provided in one embodiment of the present application;

[0044] Figure 5A A schematic flowchart of a client writing data into a database is provided for another embodiment of the present application;

[0045] Figure 5B A schematic flowchart of a client reading data from a database is provided for another embodiment of the present application;

[0046] Figure 6 A schematic interactive flow chart of a data reading method provided in one embodiment of the present application;

[0047] Figure 7 An embodiment of the present application provides a schematic flowchart of a client using an SDK to cyclically read a string S in a KV database with a reading step length of n;

[0048] Figure 8 A schematic interactive flow chart of a data reading method provided in one embodiment of the present application;

[0049] Figure 9 A schematic interactive flow chart of a data reading method provided in another embodiment of the present application;

[0050] Figure 10 A schematic diagram of the structure of the client provided in an embodiment of the present application;

[0051] Figure 11 A schematic diagram of the structure of the server provided in the embodiment of the present application;

[0052] Figure 12 A schematic diagram of the structure of the chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions provided in the embodiments of the present application are described below with reference to the accompanying drawings.

[0054] It should be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0055] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0056] An embodiment of the present application provides a data reading method, which is applied to a storage system including a client and a server, wherein the server is deployed with a database, and the client writes / reads data in the database through a database connection established with the server.

[0057] Figure 1 For a schematic diagram of the storage system provided in one embodiment of the application, see Figure 1 As shown, the storage system includes a client 100 and a server 200. The number of the client 100 and the server 200 can be one or more. It should be understood that Figure 1 The numbers of the clients 100 and servers 200 shown in the figure are for example only and do not constitute a specific limitation.

[0058] The client 100 is used to provide an interactive interface for the user or directly receive data requests. The user can enter instructions through the client, which translates them into the user's read / write requests and reads / writes the corresponding data in the database. The specific form of the client can be a web browser or a locally running application, which is not specifically limited in this application.

[0059] The server (Server) 200 is used to run various applications, such as databases, websites, games, etc. Its main functions are to process requests, store data, provide computing services, etc. The database is deployed in the server. It is a warehouse that organizes, stores and manages data according to the data structure. Users can add, intercept, update, delete and other operations on the data in the warehouse through the client. Generally, the types of databases include non-relational databases (NoSQL), such as KV databases. KV databases are a simple and efficient data storage structure that divides data into two parts: key (Key) and value (Value). The key is used to quickly locate and find the corresponding value. It includes memory-based KV databases, such as Redis, RAMCloud, etc.; disk-based databases, such as Memcached, HBase, etc.; distributed KV databases, such as Cassandra, Amazon Dynamo, etc. In actual applications, clients, servers and databases are often used together to build complex applications. For example, the client can develop an application based on a software development kit (SDK), the server can process the application's requests, and the database can store the application's data, thereby enabling the application to run on the client and data storage and management through the server and database.

[0060] In some embodiments, multiple clients 100 can access the database of the server 200 to read, update, delete, etc. data in the database. Specifically, multiple clients 100 can access the database of the server 200 separately in different time periods, or multiple clients can access the database of the server 200 simultaneously in the same time period.

[0061] In other embodiments, multiple service threads of a service process in a client 100, or multiple service threads within multiple service processes, may also access the database of the server 200 when processing data read requests. Similarly, multiple service threads may access the database of the server 200 separately during different time periods, or may access the database of the server 200 simultaneously during the same time period.

[0062] It should be noted that, in this embodiment, the client 100 can be deployed on the server 200, or on an independent server or electronic device. The independent server may not be the server 200, but another server. In other words, the client 100 and the server 200 can be integrated into a single electronic device, or they can be integrated into different electronic devices. This application does not specifically limit this.

[0063] The following uses a server-deployed database, including a KV database, as an example to describe how a client writes and reads data from the server's database using the SDK. For ease of description, writing and reading data from the server's KV database is referred to as writing and reading data from the KV database. For example, reading data from the server's KV database is referred to as reading data from the KV database.

[0064] In this embodiment, SDK is a software development kit integrated into the client, which is used to proxy business threads to connect to the KV database to execute related request commands; KV database refers to a single node or a cluster composed of multiple nodes, each of which provides a database capable of storing and reading key-value pairs. The client initializes the SDK through configuration items such as the KV database node IP, calls the SDK command interface to read data, and the SDK connects to the KV database node based on the IP, reads and writes data from the KV database, and returns it; SDK uses a connection pool to manage node connections. Every time the SDK sends a request, it needs to obtain an idle connection from the connection pool for request sending. The overall process can be seen in Figure 2A as well as Figure 2B As shown in .

[0065] It should be understood that the data type of the Value written in the KV database can be a string, a linked list, a set, an ordered set, etc. Taking a string as an example, in the KV database, one Key value corresponds to one string.

[0066] In some embodiments, when writing a string, the client calls the corresponding SDK command interface to write the string into the corresponding key value in the KV database; when reading the string, the SDK requests an idle connection from the database connection in the connection pool, and based on the requested idle connection, sends a read command to the KV database. The KV database reads all the strings stored under the key value corresponding to the read command at one time and returns them to the client through the SDK.

[0067] For example, for data writing scenarios, see Figure 3A As shown in , if the string that the client needs to write is S1, when writing the string, the client directly calls the SDK command interface to write the string S1 into the key value Key 1 corresponding to the KV database, that is, Key 1 = S1. For data reading scenarios, see Figure 2BAs shown, when the client reads the string S1, the SDK requests an idle connection and sends a read command to the KV database based on the idle connection. The read command includes an identifier that can uniquely confirm the string, such as the key value Key 1; the KV database returns the string S1 corresponding to Key 1 to the SDK based on the corresponding key value Key 1, and the SDK returns the string S1 to the client.

[0068] In this embodiment, when a client writes a string into the KV database via the SDK, the client and the corresponding SDK do not process the string in any way. Instead, they directly call the SDK command interface to execute the data writing process. When reading data, the SDK reads all the strings written in the key from the KV database at once based on the received read command and the identifier in the read command.

[0069] In the method provided in this embodiment, when the string written to the Key by the client through the SDK is large, see Figure 4 As shown in the figure, the values ​​written to Key 1 and Key 2 in the KV database are relatively small, while the value written to Key 3 is relatively large. Therefore, when multiple business clients access the same KV database during the same period, they will affect each other. If a client accesses a large amount of data, it will cause the KV database to be blocked, reducing the efficiency of the KV database in processing requests and affecting the processing of requests from other clients.

[0070] In addition, due to the limited number of database connections between the KV database and the client, during the data reading process, a large amount of time will be occupied under a certain database connection because the amount of data requested by a certain data reading request between a certain application or a certain business thread in the client and the database is large, resulting in a reduction in the number of database connections that can be shared by other applications or other business threads in the client and the database in the same period. For example, if there are a total of 10 database connections in the connection pool for 20 business threads in the client to read data, the data request 1 used by business thread 1 to read data from the database occupies one of the database connections for up to 10 minutes. During the 10 minutes, the other 19 business threads can only request idle connections from the remaining 9 database connections. It can be seen that this method causes multiple requests in the client to affect each other. If the data accessed by a certain request is large and takes too long, it will occupy the database connection for a long time, resulting in other requests being unable to obtain idle connections.

[0071] In other embodiments, when writing a string, the SDK splits the string that the client needs to write into multiple substrings based on the string size threshold setting, and stores them in multiple keys; when reading a string, the SDK requests an idle connection from the connection pool, reads each substring stored in multiple keys, and splices the substrings in the reading order and returns them to the client.

[0072] For example, for data writing scenarios, see Figure 5A As shown in , if the string that the client needs to write is S2, when writing the string, the SDK will split the string S2 into multiple substrings based on the size threshold of the string S2, for example, split S2 into S2-1, S2-2 and S2-3, and then store them in multiple keys, for example, Key 1 = S2-1, Key 2 = S2-2 and Key 3 = S2-3. In this case, when the KV database stores the multiple substrings corresponding to the string S2, it also stores the storage order of the multiple substrings in the key. For data reading scenarios, see Figure 5B As shown in , when the client reads string S1, the SDK requests an idle connection and then sends a read command to the KV database based on the idle connection to read the substrings S2-1, S2-2, S2-3 stored in Key1, Key 2, and Key 3 respectively, as well as the storage order. The substrings are then concatenated into string S2 in the reading order and returned to the client.

[0073] In this embodiment, the client splits the string through the SDK and writes it to the KV database. This requires additional storage space (the basic storage space to which the data structure belongs) to store the storage order of each substring and the entire string, which increases the storage space required for the split data and increases the cost. In addition, since multiple clients can access the database during the data reading process, in this case, if a client is reading a certain piece of data while another client is updating or deleting this data, the data read by the current client will be unreliable, resulting in problems such as dirty reads (for example, some substrings come from the updated data, and some substrings come from the data before the update) or missed reads (for example, some substrings are lost).

[0074] Based on the above problems, an embodiment of the present application provides a data reading method that obtains target data by cyclically reading received data segments. That is, in each cycle, after the client obtains an idle connection, it only reads data of a certain read step length. When it is determined that the target data has been read, the read data segments are spliced ​​to obtain the target data. This solves the problem that due to the large amount of data read in a single time, the time it takes to occupy the KV database connection is long, resulting in other requests being unable to obtain an idle connection, thereby reducing the execution efficiency of the client and KV database threads and affecting the normal reading of other data.

[0075] Figure 6 This is a schematic interactive flow chart of a data reading method provided in an embodiment of the present application. The method is applied to a storage system including a client and a server, wherein the client reads data from a KV database through N KV database connections established with the server, where N ≥ 1. Figure 6 As shown, the method includes the following steps S601 to S607.

[0076] S601: The client receives a first operation, where the first operation is used to instruct the client to read target data from a KV database.

[0077] The first operation may be a user clicking on a client operation interface, or inputting an operation instruction through the client, which is used to instruct the client to read the target data from the KV database by cyclically sending a read command. For example, the client receives a user click operation on a first application, and the click operation is used to instruct the client to execute the login process of the first application. For example, after the user enters user information in the login interface of the first application, the user clicks the "Login" control to log in to the first application. At this time, the client needs to request to read the target data related to the user information entered by the user from the KV database.

[0078] S602: The client obtains an idle connection from N KV database connections according to the first operation.

[0079] In this embodiment, the client can use any one of N KV database connections to read data from the KV database. These N KV database connections are used by any business thread within the client that needs them. Each time a business thread needs to read data from the KV database, it first obtains an idle connection from the N KV database connections.

[0080] In some embodiments, KV database connections are centrally managed and allocated by a connection pool. When a client or a thread within a client needs to perform data operations on the KV database, it requests an idle connection from the N KV database connections in the connection pool. When the thread or client has finished using the connection, it returns it to the connection pool to be used by other threads or clients that need to use it. Figure 2A as well as Figure 2B As shown in , the client sends an idle connection request to the connection pool, which is used to obtain an idle connection from the connection pool. After receiving the idle connection request sent by the client, the connection pool returns the idle connection to the client from N KV database connections.

[0081] S603: The client sends a data read request to the server through the idle connection. The data read request includes a first command, a data identifier of the target data, a first read cursor, and a read step.

[0082] The first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the first read cursor and the read step size. Exemplarily, the first command may be a scanStr command.

[0083] The data identifier of the target data refers to identification information that can uniquely identify the target data. After receiving the data identifier, the server can determine the unique target data from the KV database based on the data identifier.

[0084] The first read cursor is a non-negative integer that controls the position in the target data from which to begin reading based on the current data read request. For example, the initial value of the first read cursor can be 0, indicating that the read starts at position 0, and the maximum value can be 1844674407370955161. In subsequent loops, the second read cursor, carried in the read result returned by the previous data read request, is sent to the server as the read cursor parameter for the next loop.

[0085] In this embodiment, the read cursor of the target data can be the starting position of the target data or a position in the middle of the target data. After receiving the read cursor, the server can determine the position to start reading the target data from the KV database based on the read cursor.

[0086] The read step size refers to the length of data greater than zero requested for this data read request, also known as the data threshold. The data length requested in each data read request can be the same, for example, a read step size of 10 in each data read request. It can also be different, for example, a read step size of 8 in one data read request and 10 in the next. After receiving the read step size, the server determines the end point of reading the target data from the KV database based on the read step size and the read cursor.

[0087] For example, let's assume that the target data stored in the KV database is a string, which includes 1 to M data items, each of which has a length of 1, where M ≥ 1. The data identifier can be the key value in the KV database (when the string is a Value), or the Key plus other parameters (when the string is an element of a set Value); the read cursor can be the position of the i-th data item in the string, where 0 ≤ i ≤ M-1; and the read step size can be 10 data items.

[0088] In some embodiments, the data read request also includes a version number, such as a first version number. In this embodiment, the version number can also be called an identification number (for example, its value range can be 0-1844674407370955161), which is the identification information assigned to the stored data in the KV database. When the target data is modified or deleted, the KV database will assign a new version number to the target data. Therefore, it is possible to determine whether the target data has been modified or deleted by comparing the version numbers. Exemplarily, in the KV database, the KV database records a version number for each Key or collection element. If the data recorded in the Key changes, the version number will also change accordingly. The version number can be a timestamp, that is, after each data update, the KV database can use the current timestamp as the new version number.

[0089] It should be understood that when the client sends a data read request for the target data to the server for the first time, the first read cursor included therein may be the starting position of the target data, for example, the position when i=0, indicating that reading starts from position 0; based on the version number generated on the KV database side, the first version number in the data read request sent for the first time may be an initialization version number, for example, it may be 0.

[0090] In some embodiments, when a client sends a data read request to a server for the first time through an idle connection, the data read request includes a first command, a data identifier of the target data, a first read cursor, and a read step size, but does not include the version number of the target data. In other words, when the client first passes the parameters in the data read request to the server, it does not pass the version number information of the target data. Instead, when the second version number of the target data is identified in the read result returned by the server, the client sends the second version number together with the data read request to the server in the next cycle. It should be noted that in this embodiment, the case where the version number is 0 is attributed to including the version number, and the version number is 0.

[0091] S604: The server executes the first command according to the data read request to obtain the data segment corresponding to the data identifier.

[0092] After receiving a data read request from a client, the server logs into the KV database based on the data read request and enters the first command, the target data identifier, the first read cursor, and other information into the interactive interface provided by the KV database. The server then determines the target data from the KV database based on the data identifier in the data read request. After determining the target data, the server executes the first command, which determines the starting point for reading the target data based on the first read cursor in the data read request and the ending point for reading the target data based on the read step size. The server then generates a data segment using the data after the first read cursor and with a data length between the read step sizes.

[0093] Optionally, when the read step in the data read request is 0 or the data read request does not include a read step, the server executes a first command according to the data read request to obtain the data segment corresponding to the data identifier with a preset read step; when the read step in the data read request is not 0, the server executes the first command according to the data read request to obtain the data segment corresponding to the data identifier with the read step in the data read request.

[0094] S605: When the third preset condition is met, the server sends a reading result to the client, where the reading result includes a data segment corresponding to the target data.

[0095] The read result also includes the second read cursor of the target data in the next cycle of the client and indication information of whether all the target data has been read.

[0096] Optionally, the server may determine the second read cursor of the target data in the next cycle and whether all the target data has been read according to the position where the reading of the target data ends.

[0097] In some embodiments, the read result returned by the server also includes the second version number of the target data recorded in the KV database. Based on this, in the kth cycle, before sending a data read request to the server via an idle connection, or after the end of the k-1th cycle, the client updates the first read cursor to the second read cursor and updates the first version number to the second version number, where k ≥ 2.

[0098] As can be seen from the above, under the premise that the data read request sent by the client includes a version number, the version number in the data read request received by the server may be 0 or non-0. When the version number in the data read request is 0, it may be the first version number carried in the first data read request sent by the client to the server; when the version number in the data read request is not 0, it may be the second version number carried in other data read requests sent by the client to the server. The non-0 version number may be the one carried in the read result returned by the server to the client and sent again by the client to the server. If the version number in the data read request received by the server is 0 or the version number information of the target data is not received, the server does not verify the version number. Otherwise, the server needs to verify the version number.

[0099] In some embodiments, when the version number in the data read request is 0 or the version number information of the target data is not received, the server determines that the third preset condition is met. In this case, the server sends the read result to the client.

[0100] In other embodiments, when the version number in the data read request is not 0 and the version number in the data read request is consistent with the version number of the target data recorded in the KV database, the server determines that the third preset condition is met. In this case, the server sends the read result to the client. When the version number in the data read request is not 0 and the version number in the data read request is inconsistent with the version number of the target data recorded in the KV database, the server determines that the third preset condition is not met and generates an error message. The error message is used to indicate that the target data has changed. In this case, the server sends the error message to the client to indicate that the target data currently being read by the client has changed and stops the data reading process.

[0101] When the third preset condition is met, the server packages the data segment corresponding to the target data, the next read cursor, and the indication information of whether all the target data has been read, generates a read result, and returns it to the client.

[0102] In some embodiments, the server generates first indication information when the target data is read, and the first indication information is used to indicate that the target data has been read in full; and generates second indication information when the target data is not read, and the second indication information is used to indicate that the target data is not read.

[0103] In other embodiments, the server generates a first indication only when the target data has been read. This first indication indicates that all the target data has been read. No indication is generated when the target data has not been read. For example, this first indication can be combined with a read cursor or version number. For example, when the read cursor is 0 again, it indicates that the target data has been read.

[0104] S606: When the first preset condition is not met, the client executes the above steps S602 to S605 in a loop; when the first preset condition is met, the client executes step S607.

[0105] The first preset condition includes receiving first indication information sent by the server, where the first indication information is used to indicate that all target data has been read.

[0106] As described in the above embodiment, the reading result returned by the server includes indication information for indicating whether all target data has been read. The indication information can be first indication information for indicating that all target data has been read, or second indication information for indicating that the target data has not been read.

[0107] In this step, when the client determines that the read result does not include the first indication information or the second indication information, or includes the second indication information but not the first indication information, it indicates that the first indication information indicating that the target data has been read in full has not been received, that is, the target data has not been read completely. In this case, the client re-acquires an idle connection from the N KV database connections, and after obtaining the idle connection, sends a data read request to the server through the obtained idle connection. The data read request includes the data identifier of the target data, the second read cursor, and the read step size. It should be noted that the second read cursor in the data read request is the second read cursor in the next cycle included in the read result returned by the server in the previous cycle. Optionally, the data read request also includes the second version number of the target data. The version number is the second version number of the target data recorded in the KV database that the server packages into the read result according to the previous data read request in the previous cycle and returns to the client.

[0108] In this step, when the client determines that the reading result includes the first indication information, it indicates that all the target data has been read.

[0109] The client uses the next second data read cursor and read step returned by this data read request as the request parameters for the next data read request, continues to request the next data segment to implement a circular call, and determines whether the target data has been read by whether the first indication information is received, until all data segments are requested.

[0110] It should be noted that in this embodiment, when the client executes each loop, the data identifier of the target data in the data read request sent to the server is the same, so that when the server receives the data read request, it reads the content of the target data from the KV database each time. In addition, the read step size included in the data read request sent by the client to the server in the k-th loop is determined by the client based on the read result received in the k-1-th loop. At the same time, in the k-th loop, the client updates the first read cursor to the second read cursor and the first version number to the second version number, so that during the execution of the loop, the read cursor and version number are dynamically updated, ensuring the accuracy of the read data.

[0111] In some embodiments, the client determines the read step size in the k-th cycle based on the response time of the read result received in the k-1-th cycle. For example, if the response time of the read result received in the k-1-th cycle is longer, the read step size in the k-th cycle is reduced; if the response time of the read result received in the k-1-th cycle is shorter, the read step size in the k-th cycle is increased.

[0112] In other embodiments, the client determines the read step size in the kth cycle based on the number of read results received. For example, if the client determines that the target data has not been read completely within a preset number of times, the read step size in the kth cycle is increased to reduce the number of times the target data is read, thereby completing the target data read process as quickly as possible, thereby allowing the KV database to process data requests from other clients.

[0113] In this embodiment, when determining the reading step length in the kth cycle, the client controls each reading step length to be less than or equal to the first threshold value to avoid the problem of blocking the KV database due to the value of the reading step length being set too large.

[0114] S607: When the first preset condition is met, the client stops the loop and splices all the read data segments in the reading order to obtain the target data.

[0115] In this embodiment, when the client and the server do not transmit version number information to each other during data reading, the first preset condition includes: receiving first indication information sent by the server, where the first indication information is used to indicate that all target data has been read.

[0116] For example, when the client determines that the reading result includes the first indication information, it stops executing the loop process of steps S602 to S604 and splices the read data segments in the reading order to obtain the target data. Figure 7 As shown in , when the client uses the SDK to loop through the string S in the KV database with a reading step of n, if the data segments read by the client include S-1, S-2, S-3, S-4, and S-5, the client concatenates S-1, S-2, S-3, S-4, and S-5 in the reading order to obtain the string S, which is the target data.

[0117] When version number information is transmitted between the client and the server during the data reading process, the client needs to determine the third version number in the kth loop. The third version number is the second version number of the target data recorded in the KV database included in the reading result of the k-1th loop. At this time, the first preset condition includes: the second version number is consistent with the third version number, wherein the consistency of the second version number and the third version number is used to indicate that the target data has not changed, and the first indication information sent by the server is received, and the first indication information is used to indicate that the target data has been read in full. When the second version number and the third version number are consistent and the first indication information sent by the server is received, the client stops executing the above-mentioned loop process of steps S602 to S605 and splices the read data segments in the reading order to obtain the target data.

[0118] Optionally, the client stops the loop and deletes all read data segments when a second preset condition is met. The second preset condition includes: the second version number is inconsistent with the third version number, wherein the inconsistency between the second version number and the third version number is used to indicate that the target data has changed; or receiving an error message sent by the server, which is generated by the server in the kth loop when it determines that the version number in the received data read request is inconsistent with the version number recorded in the KV database, and the error message is used to indicate that the target data has changed.

[0119] The present application provides a data reading method that employs a split, cyclical reading method to obtain target data. After each request to obtain an idle connection, the client sends a data read request to read the target data according to a certain read step size. After receiving a first indication message from the server indicating that the target data has been read, all read data segments are concatenated to obtain the target data. This method enables the client to read only the data segments corresponding to a certain threshold each time it obtains an idle connection.

[0120] Through the method provided in this embodiment, on the one hand, when multiple clients access the same KV database, it can be ensured that they do not affect each other. The KV database can process the data reading request issued by each client accordingly, and there will be no problem that the KV database is blocked due to the large size of data accessed by a certain client, which reduces the efficiency of the KV database in processing requests and affects the request processing of other clients; on the other hand, it can reduce the impact of data reading requests corresponding to multiple business threads within the same client. There will be no problem that the data reading request of a certain business thread accesses a large amount of data, takes too long, occupies the client connection time for a long time, and causes other requests to be unable to obtain idle connections.

[0121] The data reading method provided in the embodiment of the present application is further explained below using two specific examples.

[0122] Example 1

[0123] Figure 8 This is a schematic interactive flow chart of a data reading method provided in an embodiment of the present application. The method involves a first application in a client reading target data in batches by calling an SDK segmented read command (i.e., sending a data read request in a loop) until the target data is read. Specifically, take the first application in the client reading target data (Key=string, Value=123456789009876543211234567890) and requesting a KV database through the SDK segmented interface: scanStr as an example, see Figure 8 As shown, the method includes the following steps S801 to S812.

[0124] S801: A first application receives a first operation.

[0125] Illustratively, the first operation may be a click operation on the first application to instruct the first application to read target data Value=123456789009876543211234567890 from the KV database.

[0126] S802: The first application sends a data read request to the SDK according to the first operation.

[0127] The data read request is used to request the SDK to read target data from the KV database by cyclically sending data read requests.

[0128] S803: SDK obtains an idle connection from N KV database connections according to the data read request.

[0129] S804, the SDK sends a first data read request to the server through the idle connection, where the first data read request includes a data identifier of the target data, a first read cursor, a read step length, and a first version number of the target data.

[0130] In this embodiment, the data identification Key value of the target data is key=string, the first reading cursor is cursor=0, the reading step length is count=10, and the first version number is version=0.

[0131] S805: The server reads data from the KV database according to the first data read request, generates and returns a first read result to the SDK.

[0132] The server determines that the target data is the string 123456789009876543211234567890 corresponding to string in the KV database based on the data identifier Key value string in the first read command, and determines the first data segment S-1=1234567890 to be read this time and the second read cursor cursor=10 in the next loop read command based on the first read cursor cursor=0 and the read step count=10. It should be noted that, in this embodiment, the first version number in the data reading request received by the server is 0, so it is determined that the third preset condition is met. In this case, the server directly sends the reading result to the client, that is, the server packages the first data segment S-1=1234567890, the second reading cursor cursor=10 in the next loop, the second version number version=1704253321324115 corresponding to the character string 123456789009876543211234567890, and the indication information for indicating whether all the target data has been read, generates a first reading result, and returns it to the SDK.

[0133] S806, the SDK determines whether the first reading result includes the first indication information. If not, step S806 is executed.

[0134] The first indication information can be combined with the second read cursor or the second version number. For example, when the read cursor is again 0, it is determined that the first read result includes the first indication information. In this embodiment, as described in step S805, the second read cursor in the read result is cursor = 10, so the SDK determines that the read result does not include the first indication information.

[0135] S807: The SDK determines that the first read result does not include the first indication information, and re-acquires an idle connection from the N KV database connections.

[0136] The SDK determines based on the first read result that the first read result does not include the first indication information, indicating that the target data has not been read completely. At this time, it re-acquires an idle connection from the N KV database connections to continue reading the next data segment in the target data.

[0137] S808, the SDK sends a second data read request to the server through the newly acquired idle connection, where the second data read request includes a data identifier of the target data, a second read cursor, a read step size, and a second version number of the target data.

[0138] In this embodiment, the second read command includes the target data's data identifier key value key = string, the second read cursor cursor = 10, the read step count = 10, and the second version number version = 1704253321324115. In other words, at this point, the SDK has updated the first read cursor in the data read request to the second read cursor, and the first version number to the second version number, and so on, performing similar updates in subsequent loops.

[0139] S809: The server reads data from the KV database according to the second data read request, generates and returns a second read result to the SDK.

[0140] Based on the data identifier Key value string in the second read command, the server determines that the target data is the string 123456789009876543211234567890 corresponding to string in the KV database. Based on the second read cursor cursor = 10 and the read step length count = 10, the server determines the second data segment S-2 = 0987654321 to be read, as well as the read cursor cursor = 20 in the next read command. The server then packages the second data segment S-2 = 0987654321, the next third read cursor cursor = 20, the third version number version = 1704253321324115 corresponding to the string 123456789009876543211234567890, and information indicating whether all target data has been read, generating a second read result and returning it to the SDK.

[0141] In some embodiments, the server compares the second version number in the second read command with the third version number of the target data recorded in the KV database. If they are consistent, it indicates that the target data has changed, for example, been modified or updated. If they are inconsistent, it indicates that the target data has not changed. In this case, the server directly returns an error message to the SDK, which is used to indicate that the target data has changed. In this embodiment, the server compares the second version number version=1704253321324115 in the second read command with the third version number version=1704253321324115 recorded in the KV database. The comparison result shows that the version numbers are consistent. Therefore, it is determined that the third preset condition is met and there is no need to return an error message to the SDK.

[0142] S810, when the version numbers in the first read result and the second read result are consistent, the SDK determines whether the second read result contains the first indication information. If not, steps S807 to S809 are executed in a loop until the first indication information sent by the server is received. If it is contained, step S811 is executed.

[0143] In this embodiment, the version numbers in the first and second read results are consistent, i.e., the second version number version=1704253321324115 is consistent with the third version number version=1704253321324115, indicating that the string has not changed during the SDK's cyclic reading process. At this point, the SDK further determines whether the second read result contains the first indication information. If not, steps S806 to S808 are executed repeatedly until the first indication information is received from the database, indicating that the loop is complete and the target data has been read. If so, step S810 is executed.

[0144] In some embodiments, if the SDK receives error information returned by the server, the SDK determines that the target data has changed, stops executing the data reading process, and reports an error to the first application to instruct the first application to re-read the target data.

[0145] S811, the SDK splices all the read data segments in the reading order to obtain the target data.

[0146] In this embodiment, the target data read by the first application through the SDK is the string 123456789009876543211234567890. Each data read request uses a read step of 10. If the version number of the target data in each read result is consistent, the SDK can obtain three read results. The data segments in each read result are: the first data segment S-1 = 1234567890, the second data segment S-2 = 0987654321, and the third data segment S-3 = 1234567890. The SDK concatenates these three data segments in the order they were read to obtain the target data string 123456789009876543211234567890.

[0147] S812: The SDK returns target data to the first application.

[0148] It should be noted that, in this embodiment, when the first application calls the SDK segmented data read request to read the target data from the KV database, the data read request includes the key, the initial read cursor, the initial version number and the read step (i.e., the amount of data requested in a single request); the read result returned by the KV database includes the data segment requested in the data read request, the next read cursor and the version number of the target data; the data read request in the next cycle of the SDK uses the KV database according to the read cursor and the version number of the target data in the next cycle returned by the previous data read request, and repeats the process until the first indication information is received, indicating that all data has been requested. In the process of the application reading the target data in the KV database through the SDK, it can avoid the problem of long reading time due to large data, blocking the business or KV database, smoothing the delay of the application requesting the KV database, and ensuring the stable operation of the application and the database.

[0149] The data reading method provided in the embodiment of the present application can be applied to the reading process of string type data. The first application calls the SDK batch data reading request and sends the batch data reading request to the server. The position and length of each reading are controlled by the reading cursor and the reading step size, and the data segments are read in batches until the reading is completed and the splicing is returned. This process enables the first application to read the string data according to a certain threshold size, and the read data can be trusted based on the version number control, and there is no problem of dirty reading and missed reading. In addition, through the process of reading the target data in a segmented loop, there is no need to split the string into multiple key storages, record the reading order and the additional storage space required for multiple key storages, thereby saving costs.

[0150] Example 2

[0151] Figure 9This is a schematic interactive flow chart of a data reading method provided in an embodiment of the present application. The method involves the processing logic of the target data being modified by the second application during the process of the first application in the client calling the SDK segmented read command (i.e., sending data read requests in a loop) to read the target data in batches. Specifically, take the first application in the client reading the target data (Key=string, Value=123456789009876543211234567890) and requesting the KV database through the SDK segmented interface: scanStr as an example, see Figure 9 As shown, the method includes the following steps S901 to S918.

[0152] S901: A first application receives a first operation.

[0153] S902: The first application sends a data read request to the SDK according to the first operation.

[0154] S903: The SDK obtains an idle connection from N KV database connections according to the data read request.

[0155] S904, the SDK sends a first data read request to the server through the idle connection, where the first data read request includes a data identifier of the target data, a first read cursor, a read step size, and a first version number of the target data.

[0156] S905 : The server reads data from the KV database according to the first data read request, generates and returns a first read result to the SDK.

[0157] The first read result includes the data segment corresponding to the target data, the second read cursor of the target data in the next cycle, the second version number of the target data recorded in the KV database, and indication information of whether all the target data has been read.

[0158] S906, the SDK determines whether the first reading result includes the first indication information. If not, step S909 is executed.

[0159] S907 , the second application sends a modification command to the server, where the modification command is used to instruct the server to modify the target data in the KV database.

[0160] The modification command includes the data identifier of the target data, that is, key=string.

[0161] S908 , the server modifies the target data in the KV database according to the modification command, and updates the version number of the target data after the modification of the target data is completed.

[0162] S909: The SDK obtains an idle connection from the N KV database connections.

[0163] S910, the SDK sends a second data read request to the server through the newly acquired idle connection, where the second data read request includes a data identifier of the target data, a second read cursor, a read step size, and a second version number of the target data.

[0164] S911. The server obtains the second version number of the target data in the second data read request and the third version number of the target data recorded in the KV database.

[0165] It should be understood that the second data read request sent by the client to the server carries the second version number of the updated target data, and the KV database records the third version number generated based on the modified target data.

[0166] That is to say, based on the aforementioned steps S907 to S908, the server has modified the target data in the KV database according to the modification command sent by the second application, and the KV database adaptively synchronizes the version number of the target data. Therefore, after this step, the version number of the target data has changed.

[0167] S912: After determining that the second version number of the target data in the second data read request is inconsistent with the third version number of the target data recorded in the KV database, the server returns an error message to the SDK, where the error message is used to indicate that the target data has changed.

[0168] It should be understood that in this example, the server has modified the target data in the KV database based on the modification command of the second application before receiving the second data read request, and has synchronously updated the version number of the target data. Therefore, the error message in this embodiment is generated by the server after receiving the second data read request, when it determines that the second version number of the target data in the second data read request is inconsistent with the third version number of the target data recorded in the KV database, and determines that the third preset condition is not satisfied, that is, it is generated during the second cycle. When this method is applied to other embodiments, the error message may be generated in the kth cycle when the server determines that the version number in the received data read request is inconsistent with the version number recorded in the database, where k ≥ 2.

[0169] S913: The SDK stops the loop according to the error message and deletes all read data segments.

[0170] The SDK stops executing the data reading process based on the error message and clears all read data segments.

[0171] S914, the SDK sends an error report to the first application.

[0172] The error report is used to indicate that the target data of the first application has changed, and prompt the first application to re-read the modified target data.

[0173] In this embodiment, steps S911 to S914 refer to Figure 9 As shown in Scenario 1. Optional, see Figure 9 As shown in Scenario 2, in this embodiment, steps S911 to S914 may also be:

[0174] S915: The server reads data from the KV database according to the second data read request, generates and returns a second read result to the SDK.

[0175] S916, the SDK obtains the second version number of the target data in the first read result, and the third version number of the target data in the second read result.

[0176] S917 , after the SDK determines that the second version number of the target data in the first read result is inconsistent with the third version number of the target data in the second read result, it stops the loop and deletes all read data segments.

[0177] S918. The SDK sends an error report to the first application.

[0178] The inconsistent version number is used to indicate that the target data has changed.

[0179] The data reading method provided in this embodiment involves a process in which the target data is modified by a second application during the process of a first application reading the target data through an SDK batch data read request. The first application calls the SDK batch data read request and sends a request to the server. The server controls the position and length of each read through the data read cursor and the read step size, and determines whether the data has been modified through the version number of each request. If modified, an error is returned to notify the first application that the target data has been modified. By comparing the data version numbers of the target data, the problem of dirty reading or missed reading of data is avoided. If the target data is modified by a request from another application during the reading process, the SDK will notify the current application by reporting an error or other means to prevent the current application from reading incorrect data.

[0180] It should be noted that the version number recorded in the KV database involved in the embodiment of the present application is the data version number, which is used to compare the old and new data scenarios. For example, in the master-slave node data synchronization scenario, the common KV database Redis data does not have a version number, and the master-slave data synchronization is completed by full coverage, that is, the master node sends the full data to the slave node, and the slave node unconditionally overwrites its own data; in order to improve the synchronization efficiency, before the incremental message queue is full, the synchronization data is sent in sequence based on the incremental queue. If the incremental message queue is full, the full data synchronization is triggered, and the slave node still unconditionally overwrites its own data. The KV database involved in this embodiment optimizes the master-slave data synchronization scenario based on the version number. The master node will check the data of the slave node. If the version number of a data item of the slave node is lower than that of the master node, the master node will actively push the data to the slave node for update. Conversely, the slave node will also check the data of the master node. If the version number of a data item of the master node is higher than its own, it will actively pull this data for update. When the slave node receives the synchronized data, it will also check the size of the synchronized data and its own version number, and will only receive the data if the version number is higher than its own. By comparing the version numbers and verifying the data, the master node can avoid frequent full data synchronization, which affects performance.

[0181] The relational database MySQL has two types of version numbers: one is the message version number (binlog sequence number), which is used to compare the old and new messages, but has nothing to do with the data itself. The other is the modified version number of the data, which is only used in transaction scenarios to determine whether the data can be displayed externally (if the transaction is not completed, the newly written data cannot be displayed externally), and has nothing to do with data synchronization or data freshness judgment. The version number involved in Examples 1 and 2 of this application is used to determine whether the target data has been modified, which belongs to the second type, that is, the modified version number of the data.

[0182] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0183] The embodiment of the present application also provides a client, which reads data from a KV database through N KV database connections established with the server, where N≥1, see Figure 10 As shown, the client includes: a receiving module 1001, an operating module 1002 and a processing module 1003.

[0184] The receiving module 1001 is used to receive a first operation, where the first operation is used to instruct the client to read target data from the KV database.

[0185] The operation module 1002 is configured to execute the following steps S1 to S3 in a loop according to the first operation:

[0186] S1, determine the idle connection from N KV database connections;

[0187] S2, sending a data read request to the server through the idle connection, the data read request including a first command, a data identifier of the target data, a first read cursor, and a read step length, the first command being used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length;

[0188] S3, receiving the read result returned by the server, where the read result includes the data segment corresponding to the target data, the second read cursor of the target data in the next cycle, and indication information indicating whether all the target data has been read.

[0189] The processing module 1003 is configured to stop the loop when a first preset condition is met, and to splice all the read data segments in a reading order to obtain target data.

[0190] In each loop, the data identifier of the target data is the same; the read step in the kth loop is determined by the client based on the read result received in the k-1th loop, and in the kth loop, the first read cursor is updated to the second read cursor, k≥2.

[0191] The embodiment of the present application also provides a server, which is deployed with a KV database; see Figure 11 As shown, the server includes: a receiving module 1101, an execution module 1102 and a sending module 1103.

[0192] Among them, the receiving module 1101 is used to receive a data reading request sent by the client, and the data reading request includes a first command, a data identifier of the target data, a reading cursor and a reading step. Among them, the first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, reading cursor and reading step.

[0193] The execution module 1102 is configured to execute the first command according to the data read request to obtain a data segment corresponding to the data identifier.

[0194] The sending module 1103 is used to send the reading result to the client.

[0195] The present application also provides a chip. Figure 12 As shown, the chip includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the data reading method in the above embodiments is implemented.

[0196] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the data reading method provided in the above embodiments is implemented.

[0197] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device implements the data reading method provided in the above embodiments.

[0198] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0199] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0200] In the embodiments provided in this application, the division of each framework or module is merely a logical function division. In actual implementation, there may be other division methods, for example, multiple frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0201] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0202] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0203] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0204] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A data reading method, characterized in that: Applied to a client, the client reads data from a KV database through N KV database connections established with a server, where N ≥ 1. The method includes: receiving a first operation, where the first operation is used to instruct the client to read target data from the KV database; According to the first operation, the following steps S1 to S3 are executed cyclically: S1, determining an idle connection from the N KV database connections; S2, sending a data read request to the server through the idle connection, the data read request including a first command, a data identifier of target data, a first read cursor, and a read step length, wherein the first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length; S3, receiving a read result returned by the server, wherein the read result includes a data segment corresponding to the target data, a second read cursor of the target data in the next cycle, and an indication of whether all the target data has been read; When the first preset condition is met, the loop is stopped, and all the read data segments are spliced ​​in the reading order to obtain the target data; The read step length in the kth cycle is determined by the client according to the read result received in the k-1th cycle, and in the kth cycle, the first read cursor is updated to the second read cursor, k≥2.

2. The method according to claim 1, characterized in that The read result also includes a second version number of the target data recorded in the KV database.

3. The method according to claim 2, characterized in that The data read request includes a first version number, and the method further includes: In the kth cycle, before sending the data read request to the server through the idle connection, the first version number is updated to the second version number.

4. The method according to claim 2 or 3, characterized in that The method further comprises: A third version number is determined, where the third version number is the second version number of the target data recorded in the KV database and included in the reading result of the k-1th cycle.

5. The method according to claim 4, characterized in that The first preset condition includes: The second version number is consistent with the third version number, and first indication information sent by the server is received, where the first indication information is used to indicate that all target data has been read.

6. The method according to claim 3 or 4, characterized in that The method further comprises: When the second preset condition is met, the loop is stopped and all the read data segments are deleted.

7. The method according to claim 6, characterized in that The second preset condition includes: The second version number is inconsistent with the third version number; or, The reading result includes error information, which is generated by the server in the kth cycle when it determines that the version number in the received data reading request is inconsistent with the version number recorded in the KV database. The error information is used to indicate that the target data has changed.

8. The method according to any one of claims 1 to 7, characterized in that The process of determining the read step length in the k-th cycle includes: Determined based on the response time of the read result received in the k-1th cycle; or, Determined by the number of read results received.

9. The method according to any one of claims 1 to 8, characterized in that The reading step length is smaller than or equal to a first threshold.

10. A data reading method, characterized in that: Applied to a server deployed with a KV database, the method includes: Receive a data read request sent by a client, the data read request including a first command, a data identifier of target data, a read cursor, and a read step, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step; Executing the first command according to the data read request to obtain the data segment corresponding to the data identifier; When a third preset condition is met, a reading result is sent to the client, where the reading result includes a data segment corresponding to the target data.

11. The method according to claim 10, characterized in that The data read request also includes a version number of the target data.

12. The method according to claim 11, characterized in that The third preset condition includes: The version number is 0; or, The version number is not 0, and the version number is consistent with the version number of the target data recorded in the KV database.

13. The method according to claim 11 or 12, characterized in that The method further comprises: When the third preset condition is not met, generating an error message, wherein the error message is used to indicate that the target data has changed; The error information is sent to the client.

14. The method according to any one of claims 10 to 13, characterized in that The step of executing the first command to obtain the data segment corresponding to the data identifier according to the data read request includes: When the read step size is 0, executing the first command according to the data read request to obtain the data segment corresponding to the data identifier with a preset read step size; When the read step size is not 0, the first command is executed according to the data read request to obtain the data segment corresponding to the data identifier with the read step size.

15. A client, characterized in that: The client reads data from the KV database through N KV database connections established with the server, where N≥1. The client includes: A receiving module, configured to receive a first operation, wherein the first operation is configured to instruct the client to read target data from the KV database; The operation module is configured to cyclically execute the following steps S1 to S3 according to the first operation: S1, determining an idle connection from the N KV database connections; S2, sending a data read request to the server through the idle connection, the data read request including a first command, a data identifier of target data, a first read cursor, and a read step length, wherein the first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length; S3, receiving a read result returned by the server, wherein the read result includes a data segment corresponding to the target data, a second read cursor of the target data in the next cycle, and an indication of whether all the target data has been read; A processing module, configured to stop the loop when a first preset condition is met, and to splice all the read data segments in a reading order to obtain the target data; The read step length in the kth cycle is determined by the client according to the read result received in the k-1th cycle. In the kth cycle, the first read cursor is updated to the second read cursor, and k≥2.

16. A server, characterized in that: The server is deployed with a KV database, and the server includes: a receiving module, configured to receive a data read request sent by a client, wherein the data read request includes a first command, a data identifier of the target data, a read cursor, and a read step length, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length; an execution module, configured to execute the first command according to the data read request to obtain a data segment corresponding to the data identifier; The sending module is used to send the reading result to the client.

17. A storage system, characterized in that: Includes the client according to claim 15 and the server according to claim 16.

Images