Data reading method, client, server and storage system

Abstract

A data reading method, a client, a server and a storage system. The data reading method is applied to a client, the method comprising: receiving a first operation; on the basis of the first operation, cyclically executing the following steps S1 to S3: S1, determining an idle connection from among N KV database connections; S2, sending a data reading request to a server by means of the idle connection, the data reading request comprising a first command, a data identifier of target data, a first reading cursor and a reading step length, and the first command being used for instructing the server to read, on the basis of the data identifier, the first reading cursor and the reading step length, from a KV database a data segment corresponding to the data identifier; and S3, receiving a reading result returned by the server, the reading result comprising the data segment corresponding to the target data, a second reading cursor of the target data in the next cycle, and indication information of whether the target data has been completely read; and, when a first preset condition is met, stopping the cycles, and concatenating all of the read data segments in a reading order to obtain the target data, wherein the reading step length in the k-th cycle is determined by the client on the basis of the reading result received in the (k-1)-th cycle and, in the k-th cycle, the first reading cursor is updated to the second reading cursor, k≥2.

Description

Data reading method, client, server and storage system

[0001] The present application claims priority from the Chinese patent application No. 202410437143.2 filed on April 11, 2024, and entitled "A data reading method, client, server and storage system", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of database, in particular to a data reading method, client, server and storage system. BACKGROUND

[0003] Redis is a widely used Key-Value database (KV database for short), which uses key-value pairs as storage objects and supports storing multiple data types of Value values. In Redis, each key Key corresponds to a unique Value value, and the client can obtain the corresponding Value value through the Key, which is commonly used in cache and other use scenarios.

[0004] As a common data type, string can be directly used as a Value value for storage when it is stored in a KV database, or it can be used as the smallest element unit of a Value value when the Value value is a set. However, when reading data from a KV database, the KV database or the client may be blocked during the reading process due to the large value of some strings, thereby reducing the efficiency of the KV database and / or the client in processing requests. SUMMARY

[0005] The present application provides a data reading method, client, server and storage system, which solves the problem of low efficiency of the KV database and / or the client in processing requests in the prior art.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, a data reading method is provided, which is applied to a client and used to read data from a KV database through N KV database connections established between the client and a server, where N is greater than or equal to 1. The method comprises the following steps: receiving a first operation, where the first operation is used to instruct the client to read target data from the KV database; according to the first operation, cyclically performing the following steps S1-S3: S1, determining an idle connection from the N KV database connections; S2, sending a data reading message to the server through the idle connection, where the data reading message comprises a first command, a data identifier of the target data, a first reading cursor and a reading step, the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the reading cursor and the reading step; S3, receiving a reading result returned by the server, where the reading result comprises the data segment corresponding to the target data, a reading cursor of the target data in a next cycle and indication information indicating whether the target data is completely read; when a first preset condition is met, stopping the cycle and splicing all the data segments in a reading order to obtain the target data; wherein, in each cycle, the data identifier of the target data is the same; the reading step in the kth cycle is determined by the client according to the reading result received in the (k-1)th cycle, and in the kth cycle, the first reading cursor is updated to a second reading cursor, where k is greater than or equal to 2.

[0008] Optionally, the reading step in the data reading message can be 0 or the data reading message does not comprise the reading step. When the data reading message does not comprise the reading step or the reading step in the data reading message is 0, the server executes the first command to obtain the data segment corresponding to the data identifier according to the data reading message with a preset reading step, where the preset reading step can be a first threshold, such as 250 KB.

[0009] In the method provided in the embodiment, the client can obtain the target data by cyclically reading the received data segment, that is, in each cycle, the client reads only a certain reading step of data after obtaining the idle connection, and splices the read data segments to obtain the target data when it is determined that the target data is read completely. In this way, the problem that the single reading data is large, the time-consuming of occupying the KV database connection is long, other requests cannot obtain the idle connection, the execution efficiency of the client and the KV database thread is reduced, and the normal reading of other data is affected can be solved.

[0010] In some embodiments, the reading result further comprises a second version number of the target data recorded in the KV database.

[0011] In some embodiments, the data reading request comprises a first version number, and the method further comprises: in the kth cycle, updating the first version number to a second version number before the data reading request is sent to the server through the idle connection.

[0012] In some embodiments, the method further includes: determining a third version number, the third version number being a second version number of the target data recorded in the KV database included in the read result of the k-1th cycle.

[0013] In this embodiment, when returning the read result to the client, the server returns the second version number of the target data recorded in the KV database to the client at the same time of returning the data segment corresponding to the target data, the second read cursor of the target data in the next cycle and the indication information of whether the target data is read completely, so that the client updates the first version number to the second version number before sending the data read message to the server through the idle connection in the kth cycle, or after the kth cycle ends; at the same time, the third version number is determined. In this way, the client can determine whether the target data is modified according to whether the version numbers included in the two continuous read results are consistent; in addition, the server can also determine whether the target data is modified when determining that the version number in the received data read message is inconsistent with the version number recorded in the KV database.

[0014] In some embodiments, the first preset condition includes: the second version number is consistent with the third version number, wherein the second version number being consistent with the third version number is used to indicate that the target data has not changed, and the first indication information sent by the server is received, the first indication information being used to indicate that the target data has been read completely.

[0015] In this embodiment, when the second version number is consistent with the third version number, it indicates that the target data has not changed in the data reading process; at the same time, the client receives the first indication information sent by the server, which indicates that the target data has been read completely, at this time, the client stops the cycle and splices all the data segments read according to the reading order to obtain the target data.

[0016] In some embodiments, the method further includes: when the second preset condition is met, stopping the cycle and deleting all the data segments read.

[0017] In some embodiments, the second preset condition includes: the second version number is inconsistent with the third version number, wherein the second version number being inconsistent with the third version number is used to indicate that the target data has changed; or, the error information sent by the server is received, the error information being generated by the server when determining that the version number in the received data read message is inconsistent with the version number recorded in the KV database in the kth cycle, the error information being used to indicate that the target data has changed.

[0018] When the second version number of the client is inconsistent with the third version number, or when the client receives error information sent by the server, indicating that the target data in the KV database has changed, the client stops the loop and deletes all the read data segments, thereby avoiding the problem of dirty reading or missing reading.

[0019] In some embodiments, the determination of the read step length in the kth loop includes: determining according to the response time of the read result received in the (k-1)th loop; or determining according to the number of read results received.

[0020] In this embodiment, the client determines the read step length in the kth loop according to the read result received in the (k-1)th loop, so that the read step length in each loop can be adjusted to adapt to the actual reading process, thereby improving the data reading efficiency.

[0021] In some embodiments, the read step length is less than or equal to a first threshold.

[0022] In this embodiment, when determining the read step length in the kth loop, the client controls each read step length to be less than or equal to a first threshold, so as to avoid the problem of KV database blocking caused by the read step length being set too large.

[0023] It should be noted that the read step length of the target data in the first data read message sent by the client to the server through the idle connection after obtaining the idle connection can be a preset step length, that is, when k=1, the read step length is the preset step length.

[0024] In a second aspect, a data reading method is provided, which is applied to a server deploying a KV database. The method is applied to the server reading data from the KV database by executing a database operation command. The method includes: receiving a data read message sent by a client, the data read message including a first command, a data identifier of target data, a read cursor and a read step length, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor and the read step length; executing the first command according to the data read message to obtain the data segment corresponding to the data identifier; and when a third preset condition is met, sending a read result including the data segment of the target data to the client.

[0025] In this embodiment, after receiving the data read message, the server executes the first command in the data read message to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor and the read step length, and returns the read result including the data segment of the target data to the client when the third preset condition is met.

[0026] The read result further includes a second read cursor of the target data in a next loop performed by the client and indication information of whether the target data is read completely.

[0027] In some embodiments, the data read message further includes a version number of the target data.

[0028] The version number can be 0 or not 0.

[0029] In some embodiments, the third preset condition includes that the version number is 0 or the version number is not 0 and the version number is consistent with a version number of the target data recorded in the KV database.

[0030] In this embodiment, when the version number in the data read message is 0, the server does not need to verify the version number, determines that the third preset condition is met, and sends the read result to the client; when the version number in the data read message is not 0, the version number needs to be verified.

[0031] In some embodiments, the method further includes generating error information when the third preset condition is not met, the error information being used to indicate that the target data has changed; and sending the error information to the client.

[0032] In this embodiment, when the version number in the data read message is not 0 and the version number in the data read message is inconsistent with the version number of the target data recorded in the KV database, it indicates that the target data has changed, it is determined that the third preset condition is not met, error information is generated and sent to the client to indicate the client to stop the reading process of the target data, thereby avoiding the problem of dirty reading or missing reading of the client.

[0033] In some embodiments, according to the data read message, the first command is executed to obtain the data segment corresponding to the data identifier, including: when the reading step is 0, according to the data read message, the first command is executed to obtain the data segment corresponding to the data identifier with a preset reading step; when the reading step is not 0, according to the data read message, the first command is executed to obtain the data segment corresponding to the data identifier with the reading step.

[0034] Optionally, if the reading step is not included in the data read message, the server executes the first command to obtain the data segment corresponding to the data identifier with a preset reading step according to the data read message.

[0035] In a third aspect, a client is provided, which reads data from a KV database through N KV database connections established with a server, N≥1, and the client comprises: a receiving module configured to receive a first operation, the first operation being used to instruct the client to read target data from the KV database; an operation module configured to cyclically execute the following steps S1-S3 according to the first operation: S1, determining an idle connection from the N KV database connections; S2, sending a data reading message to the server through the idle connection, the data reading message comprising a first command, a data identifier of the target data, a reading cursor and a reading step, the first command being used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the first reading cursor and the reading step; S3, receiving a reading result returned by the server, the reading result comprising a data segment corresponding to the target data, a second reading cursor of the target data in the next cycle and indication information of whether the target data is completely read; and a processing module configured to stop the cycle and splice all the data segments in the reading order to obtain the target data when a first preset condition is met; wherein, in each cycle, the data identifier of the target data is the same; the reading step in the kth cycle is determined by the client according to the reading result received in the (k-1)th cycle, and the first reading cursor is updated to the second reading cursor in the kth cycle, k≥2.

[0036] In a fourth aspect, a server is provided, which is deployed with a KV database, and the server comprises: a receiving module configured to receive a data reading message sent by a client, the data reading message comprising a first command, a data identifier of target data, a reading cursor and a reading step, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the reading cursor and the reading step; an execution module configured to execute the first command to obtain the data segment corresponding to the data identifier according to the data reading message; and a sending module configured to send a reading result to the client.

[0037] In a fifth aspect, a storage system is provided, which comprises the client in the third aspect and the server in the fourth aspect.

[0038] It can be understood that the beneficial effects of the third aspect to the fifth aspect can be referred to the related description in the first aspect to the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0039] FIG. 1 is a structural schematic diagram of a storage system according to an embodiment of the present application;

[0040] FIG. 2A is a schematic flowchart of reading data from a KV database by different business threads in the same client according to an embodiment of the present application;

[0041] FIG. 2B is a schematic flowchart of reading data from a KV database by different clients according to an embodiment of the present application;

[0042] FIG. 3A is a schematic diagram of writing data into a database by a client according to an embodiment of the present application;

[0043] FIG. 3B is a schematic diagram of reading data from a database by a client according to an embodiment of the present application;

[0044] FIG. 4 is a schematic diagram of comparing data size written by different Key values in a KV database according to an embodiment of the present application;

[0045] FIG. 5A is a schematic flowchart of writing data into a database by a client according to another embodiment of the present application;

[0046] FIG. 5B is a schematic flowchart of reading data from a database by a client according to another embodiment of the present application;

[0047] FIG. 6 is a schematic interactive flowchart of a data reading method according to an embodiment of the present application;

[0048] FIG. 7 is a schematic flowchart of a client reading a string S in a KV database by a read step n according to an embodiment of the present application;

[0049] FIG. 8 is a schematic interactive flowchart of a data reading method according to an embodiment of the present application;

[0050] FIG. 9 is a schematic interactive flowchart of a data reading method according to another embodiment of the present application;

[0051] FIG. 10 is a schematic structural diagram of a client according to an embodiment of the present application;

[0052] FIG. 11 is a schematic structural diagram of a server according to an embodiment of the present application;

[0053] FIG. 12 is a schematic structural diagram of a chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solutions provided by the embodiments of the present application will be described below with reference to the accompanying drawings.

[0055] It should be understood that, in the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein only represents a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone.

[0056] In the embodiments, the terms "first", "second", "third", etc. are used only for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly pointing to the number of technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0057] The embodiments of the present application provide a data reading method, which is applied to a storage system including a client and a server, wherein the server is deployed with a database, and the client writes / reads data in the database through a database connection established with the server.

[0058] FIG. 1 is a structural schematic diagram of a storage system according to an embodiment of the present application. As shown in FIG. 1, the storage system includes a client 100 and a server 200. The number of the client 100 and the server 200 can be one or more. It should be understood that the number of the client 100 and the server 200 shown in FIG. 1 is only for example, and cannot be regarded as a specific limitation.

[0059] The client 100 is configured to provide an interactive interface for a user or directly receive a data request. The user can input an instruction through the client, and the client translates the instruction into a read / write request of the user, and reads / writes corresponding data in the database. The specific form of the client can be a web browser or an application program running locally, and the present application does not make a specific limitation.

[0060] The server 200 is used to run various applications, such as databases, websites, games, etc. Its main functions are to process requests, store data, provide computing services, etc. The database is deployed in the server, which is a warehouse for organizing, storing and managing data according to data structure. Users can add, intercept, update, delete, etc. The data in the warehouse can be operated through the client. Generally, the types of databases include NoSQL, such as KV database. KV database is a simple and efficient data storage structure, which divides data into key and value, and quickly locates and finds the corresponding value through the key. It includes in-memory KV database such as Redis, RAMCloud, etc.; disk-based database such as Memcached, HBase, etc.; distributed KV database such as Cassandra, Amazon Dynamo, etc. In practical applications, the client, server and database are usually used together to build complex applications. For example, the client can develop applications based on software development kit (SDK), the server can process application requests, and the database can store application data, so that the application runs on the client, and the storage and management of data are realized through the server and the database.

[0061] In some embodiments, multiple clients 100 can access the database of the server 200 to read, update, delete, etc. The data in the database. Specifically, multiple clients 100 can access the database of the server 200 at different time periods, or multiple clients can access the database of the server 200 at the same time period.

[0062] In other embodiments, multiple business threads of a business process in a client 100, or multiple business threads in multiple business processes can also access the database of the server 200 when processing data reading requests. Similarly, multiple business threads can access the database of the server 200 at different time periods, or multiple business threads can access the database of the server 200 at the same time period.

[0063] It should be noted that in this embodiment, the client 100 can be deployed on the server 200, or on a separate server or electronic device. The separate server can not be the server 200, but other servers. That is, the client 100 and the server 200 can be integrated on one electronic device, or can be integrated on different electronic devices, which is not limited in the present application.

[0064] The following describes the process of writing / reading data in the database deployed in the server, taking the KV database as an example. For ease of description, writing / reading data in the KV database of the server is referred to as writing / reading data in the KV database, for example, reading data in the KV database of the server is referred to as reading data in the KV database of the server.

[0065] In this embodiment, the SDK is a software development kit integrated into the client, which is used to connect the KV database through a business thread proxy to execute related request commands; the KV database refers to a single node or a cluster composed of multiple nodes, each node provides a database capable of storing and reading key-value pairs. The client initializes the SDK through configuration items such as KV database node IP, calls the SDK command interface to read data, and the SDK reads and writes data from the KV database based on IP connection and returns; the SDK uses a connection pool to manage node connections, and each time the SDK sends a request, it needs to obtain an idle connection from the connection pool for request sending, and the overall process can be referred to as shown in FIG. 2A and FIG. 2B.

[0066] It should be understood that the data type of the Value value written in the KV database can be a string, a list, a set, and a sorted set (zset), etc. Taking a string as an example, in the KV database, a Key value corresponds to a stored string.

[0067] In some embodiments, when writing a string, the client calls the corresponding SDK command interface to write the string into the corresponding key value of the KV database; when reading a string, the SDK requests an idle connection from the database connection of the connection pool, and based on the idle connection obtained, sends a read command to the KV database, and the KV database reads all the strings stored under the corresponding key value according to the read command and returns them to the client through the SDK.

[0068] For example, for the data writing scenario, as shown in FIG. 3A, if the string that the client needs to write is S1, when writing the string, the client directly calls the SDK command interface to write the string S1 into the corresponding key value Key 1 of the KV database, i.e. Key 1 = S1. For the data reading scenario, as shown in FIG. 2B, when the client reads the string S1, the SDK requests an idle connection, and based on the idle connection, sends a read command to the KV database, which includes an identifier that can uniquely identify the string, such as the key value Key 1; the KV database returns the string S1 corresponding to the key value Key 1 to the SDK based on the corresponding key value Key 1, and the SDK returns the string S1 to the client.

[0069] In this embodiment, when the client writes a string into the KV database through the SDK, the client and the corresponding SDK do not process the string, but directly call the SDK command interface to execute the data writing process. When data is read, the SDK reads all the strings written in the Key from the KV database at one time based on the received read command and according to the identification in the read command.

[0070] In the method provided in this embodiment, when the string written into the Key by the client through the SDK is large, referring to FIG. 4, the Value values written in Key 1 and Key 2 in the KV database are relatively small, and the Value value written in Key 3 is relatively large. Therefore, when multiple business clients access the same KV database in the same period, they will affect each other. If the data size accessed by a certain client is large, the KV database is blocked, the processing request efficiency of the KV database is reduced, and the request processing of other clients is affected.

[0071] In addition, because the number of database connections between the KV database and the client is limited, in the data reading process, a certain application or a certain business thread in the client and a certain data reading request between the database will occupy a large amount of time under a certain database connection, which reduces the number of database connections that can be shared by other applications or other business threads in the client and the database in the same period. For example, if there are a total of 10 database connections in the connection pool for 20 business threads in the client to read data, data request 1 used by business thread 1 to read data from the database occupies one of the database connections for 10 minutes. During the 10 minutes, the other 19 business threads can only request idle connections from the remaining 9 database connections. As can be seen, this method makes multiple requests in the client affect each other. If a certain request accesses large data, it takes too long and occupies the database connection for a long time, which prevents other requests from obtaining idle connections.

[0072] In some other embodiments, when a string is written, the SDK sets a string size threshold to split the string needed to be written by the client into multiple sub-strings and stores the sub-strings in multiple Keys; when a string is read, the SDK reads each sub-string stored in multiple Keys after obtaining an idle connection from the connection pool, and splices and returns the sub-strings to the client in the reading order.

[0073] For example, for a data writing scenario, as shown in FIG. 5A, if the string to be written by the client is S2, when the string is written, the SDK splits the string S2 into multiple sub-strings according to the size threshold setting of the string S2, for example, S2 is split into S2-1, S2-2 and S2-3, and then the sub-strings are stored in multiple keys, for example, Key 1 = S2-1, Key 2 = S2-2 and Key 3 = S2-3. In this case, when the KV database stores the multiple sub-strings corresponding to the string S2, the storage order of the multiple sub-strings in the keys is also stored. For a data reading scenario, as shown in FIG. 5B, when the client reads the string S1, the SDK sends a reading command to the KV database based on the idle connection to read the sub-strings S2-1, S2-2 and S2-3 stored in Key 1, Key 2 and Key 3 respectively and the storage order, and then splices the sub-strings into the string S2 according to the reading order and returns the string S2 to the client.

[0074] In this embodiment, the client splits the string to write into the KV database through the SDK, which requires additional storage space (basic storage space where the data structure belongs) to store each sub-string and the storage order of the entire string, thereby increasing the storage space required after splitting and increasing the cost. In addition, since multiple clients can access the database during data reading, if a client reads a piece of data while other clients update or delete the data, the data read by the current client will be unreliable, resulting in dirty reading (for example, some sub-strings come from updated data and some sub-strings come from pre-updated data) or missing reading (for example, some sub-strings are missing).

[0075] Based on the above problems, the data reading method provided in the embodiments of the present application obtains target data by cyclically reading received data segments, that is, in each cycle, the client obtains an idle connection and reads data of a certain reading step length, and when it is determined that the target data reading is completed, the read data segments are spliced to obtain the target data. The problem that the single reading data is large, the time-consuming of occupying the KV database connection is long, other requests cannot obtain idle connections, thereby reducing the execution efficiency of the client and the KV database thread, and affecting the normal reading of other data is solved.

[0076] FIG. 6 is a schematic interaction flowchart of the data reading method provided in the embodiments of the present application, which is applied to a storage system including a client and a server, wherein the client reads data from a KV database through N KV database connections established between the client and the server, and N≥1. As shown in FIG. 6, the method includes the following steps S601-S607.

[0077] S601, the client receives a first operation, which is used to instruct the client to read target data from the KV database.

[0078] The first operation can be a user clicking on a client operation interface or inputting an operation instruction through the client, which is used to instruct the client to read the target data from the KV database by sending a read command in a loop. For example, the client receives a user click operation on the first application, which is used to instruct the client to perform a login process of the first application. For example, after the user inputs user information in the login interface of the first application, the user clicks the “login” control to log in to the first application. At this time, the client needs to request to read the target data related to the user information input by the user from the KV database.

[0079] S602, the client obtains an idle connection from the N KV database connections according to the first operation.

[0080] In this embodiment, the client can use any one of the N KV database connections to read data from the KV database. The N KV database connections are used by any business thread in the client that needs them. When a business thread needs to read data from the KV database each time, it first obtains an idle connection from the N KV database connections.

[0081] In some embodiments, the KV database connections are uniformly managed and allocated by a connection pool. When the client or a thread in the client needs to perform data operations on the KV database, an idle connection is requested from the N KV database connections in the connection pool. When the thread or the client finishes using the connection, it is returned to the connection pool to be used by other threads or clients that need to use it. For example, in combination with FIG. 2A and FIG. 2B, the client sends an idle connection request to the connection pool, which is used to obtain an idle connection from the connection pool. After receiving the idle connection request sent by the client, the connection pool returns an idle connection to the client from the N KV database connections.

[0082] S603, the client sends a data read request to the server through the idle connection, the data read request including a first command, a data identifier of the target data, a first read cursor and a read step length.

[0083] The first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the first read cursor and the read step length. For example, the first command can be a scanStr command.

[0084] The data identifier of the target data refers to identifier information that can uniquely identify the target data. After receiving the data identifier, the server can determine the unique target data from the KV database according to the data identifier.

[0085] The first read cursor refers to the position in the target data based on which the target data is read this time, is a non-negative integer, and is a parameter for controlling the position of the target data read. For example, the initial value of the first read cursor can be 0, indicating that the data is read from the 0th position, and the maximum value can be 1844674407370955161. In subsequent loops, the second read cursor carried in the read result returned by the last data read request is used as the read cursor parameter in the next loop to send to the server.

[0086] In this embodiment, the read cursor of the target data can be the starting position of the target data, or a certain position in the middle of the target data. After receiving the read cursor, the server can determine the position in the KV database from which the target data is read.

[0087] The read step refers to the data length greater than zero requested by the current data read request, which can also be referred to as a data threshold. The data length requested in each data read request can be the same, such as the read step included in each data read request being 10; or different, such as the read step included in the current data read request being 8 and the read step included in the next data read request being 10. After receiving the read step, the server can determine the position in the KV database at which the target data is read according to the read step and the read cursor.

[0088] For example, the target data stored in the KV database is a string, the string includes 1-M data, the length of each data is 1, and M≥1. The data identifier can be the key value Key (when the string is Value) in the KV database, or the Key plus other parameters (when the string is an element of a certain set Value); the read cursor can be the position of the ith data in the string, where 0≤i≤M-1; and the read step can be 10 data.

[0089] In some embodiments, the data read request further includes a version number, such as a first version number. In this embodiment, the version number can also be referred to as an identification number (for example, its value range can be 0-1844674407370955161), which is identification information given to the stored data in the KV database. When the target data is modified or deleted, the KV database will give the target data a new version number, so that whether the target data is modified or deleted can be determined by comparing the version numbers. For example, in the KV database, the KV database records a version number for each Key or collection element. If the data recorded in the Key changes, the version number will also change. The version number can be a timestamp, that is, after each data update, the KV database can use the current timestamp as a new version number.

[0090] It should be understood that when the client first sends a data read request to the server for the target data, the first read cursor included therein can be the starting position of the target data, such as the position when i = 0, indicating reading from the 0 position; the version number is generated based on the KV database side, so the first version number in the data read request sent for the first time can be an initialization version number, such as 0.

[0091] In some embodiments, when the client first sends a data read request to the server through an idle connection, the data read request includes a first command, a data identifier of the target data, a first read cursor, and a read step, but does not include a version number of the target data. That is, the client does not transmit the version number information of the target data when transmitting the parameters in the data read request to the server for the first time. Instead, when the second version number of the target data is identified in the read result returned by the server, the second version number is transmitted to the server together with the data read request in the next loop. It should be noted that in this embodiment, the case where the version number is 0 is attributed to the case where the version number is included, which is 0.

[0092] S604, the server executes the first command to obtain the data segment corresponding to the data identifier according to the data read request.

[0093] After receiving the data read request sent by the client, the server logs in the KV database according to the data read request, inputs the first command, the data identifier of the target data, the first read cursor, etc. in the interactive interface provided by the KV database, and then determines the target data from the KV database according to the data identifier in the data read request. After determining the target data, the server executes the first command, that is, determines the position to start reading the target data according to the first read cursor in the data read request, and then determines the position to end reading the target data according to the read step. The server generates a data segment with a data length between the first read cursor and the read step.

[0094] Optionally, when the read step length in the data read request is 0 or the data read request does not include the read step length, the server executes the first command to obtain the data segment corresponding to the data identifier according to the preset read step length based on the data read request; when the read step length in the data read request is not 0, the server executes the first command to obtain the data segment corresponding to the data identifier according to the read step length in the data read request.

[0095] S605, the server sends the read result to the client when the third preset condition is met, and the read result includes the data segment corresponding to the target data.

[0096] The read result further includes the second read cursor of the target data in the next loop and the indication information of whether the target data is read completely.

[0097] Optionally, the server can determine the second read cursor of the target data in the next loop and whether the target data is read completely based on the position of the target data.

[0098] In some embodiments, the read result returned by the server further includes the second version number of the target data recorded in the KV database. Based on this, the client updates the first read cursor to the second read cursor and updates the first version number to the second version number before sending the data read request to the server through the idle connection in the kth loop or after the end of the k-1th loop, where k≥2.

[0099] As can be seen from the above, under the premise that the data read request sent by the client includes the version number, the version number in the data read request received by the server can be 0 or not 0. When the version number in the data read request is 0, it can be the first version number carried in the first data read request sent by the client to the server. When the version number in the data read request is not 0, it can be the second version number carried in other data read requests sent by the client to the server, which can be carried in the read result returned by the server to the client and sent to the server again by the client. When the version number in the data read request received by the server is 0 or the version number information of the target data is not received, the server does not verify the version number, otherwise, the server needs to verify the version number.

[0100] In some embodiments, when the version number in the data read request is 0 or the version number information of the target data is not received, the server determines that the third preset condition is met, and in this case, the server sends the read result to the client.

[0101] In some embodiments, the server generates first indication information when the target data is read completely, the first indication information being used to indicate that the target data has been read completely; and the server generates second indication information when the target data is not read completely, the second indication information being used to indicate that the target data has not been read completely.

[0102] When the third preset condition is met, the server packs the data segment corresponding to the target data, the next read cursor and the indication information indicating whether the target data is read completely, and generates a read result to return to the client.

[0103] In some embodiments, the server generates first indication information when the target data is read completely, the first indication information being used to indicate that the target data has been read completely; and the server generates second indication information when the target data is not read completely, the second indication information being used to indicate that the target data has not been read completely.

[0104] In some embodiments, the server generates first indication information when the target data is read completely, the first indication information being used to indicate that the target data has been read completely; and the server generates second indication information when the target data is not read completely, the second indication information being used to indicate that the target data has not been read completely.

[0105] S606, the client cyclically executes the above steps S602-S605 when the first preset condition is not met, and executes step S607 when the first preset condition is met.

[0106] The first preset condition includes receiving the first indication information sent by the server, the first indication information being used to indicate that the target data has been read completely.

[0107] As described in the foregoing embodiments, the read result returned by the server includes indication information indicating whether the target data is read completely, which can be first indication information indicating that the target data has been read completely, or second indication information indicating that the target data has not been read completely.

[0108] In this step, when the client determines that the first indication information is not included in the reading result and the second indication information is not included, or the second indication information is included but the first indication information is not included, it indicates that the first indication information indicating that the target data has been read completely has not been received, that is, the target data is not read completely. In this case, the client reacquires an idle connection from the N KV database connections, and after acquiring the idle connection, sends a data reading request to the server through the acquired idle connection, and the data reading request includes the data identifier of the target data, the second reading cursor and the reading step. It should be noted that the second reading cursor in the data reading request is the second reading cursor in the next cycle included in the reading result returned by the server in the last cycle. Optionally, the data reading request further includes the second version number of the target data, which is the second version number of the target data recorded in the KV database packaged by the server into the reading result according to the last data reading request and returned to the client.

[0109] In this step, when the client determines that the first indication information is included in the reading result, it indicates that the target data has been read completely.

[0110] The client uses the next second reading cursor and the reading step returned by the current data reading request as the request parameters of the next data reading request, continues to request the next data segment, realizes the cyclic call, and determines whether the target data is read completely by whether the first indication information is received, until all data segments are requested.

[0111] It should be noted that in this embodiment, the data identifier of the target data in the data reading request sent by the client to the server is the same when each cycle is executed, so that the server reads the content in the target data from the KV database each time when the data reading request is received. In the kth cycle, the reading step included in the data reading request sent by the client to the server is determined by the client according to the reading result received in the k-1th cycle. At the same time, the client updates the first reading cursor to the second reading cursor and updates the first version number to the second version number in the kth cycle, so that the reading cursor and the version number are dynamically updated in the process of executing the cycle, ensuring the accuracy of reading data.

[0112] In some embodiments, the client determines the reading step in the kth cycle according to the response time of the reading result received in the k-1th cycle. For example, if the response time of the reading result received in the k-1th cycle is longer, the reading step in the kth cycle is reduced; if the response time of the reading result received in the k-1th cycle is shorter, the reading step in the kth cycle is increased.

[0113] In some embodiments, the client determines the read step size in the kth cycle according to the number of read results received. For example, if the client determines that the target data has not been read completely within a preset number of times, the client increases the read step size in the kth cycle to reduce the number of read times of the target data, so as to complete the reading of the target data as soon as possible, thereby enabling the KV database to process data requests of other clients.

[0114] In the embodiment, when the client determines the read step size in the kth cycle, the client controls each read step size to be less than or equal to a first threshold, so as to avoid the problem that the KV database is blocked due to the read step size being set to a value that is too large.

[0115] S607, the client stops the cycle when the first preset condition is met, and splices all the data segments read according to the read order to obtain the target data.

[0116] In the embodiment, when the client and the server do not exchange version number information during the data reading process, the first preset condition includes: receiving first indication information sent by the server, the first indication information being used to indicate that the target data has been read completely.

[0117] For example, when the client determines that the read result includes the first indication information, the client stops the cycle process of steps S602-S604, and splices the data segments read according to the read order to obtain the target data. For example, referring to FIG. 7, when the client reads the string S in the KV database by using the SDK with a read step size of n, if the data segments read by the client include S-1, S-2, S-3, S-4 and S-5, the client splices S-1, S-2, S-3, S-4 and S-5 according to the read order to obtain the string S, i.e., the target data.

[0118] When the client and the server exchange version number information during the data reading process, the client needs to determine a third version number in the kth cycle, the third version number being the second version number of the target data recorded in the KV database and included in the read result of the k-1th cycle. At this time, the first preset condition includes: the second version number is consistent with the third version number, and the client receives first indication information sent by the server, the first indication information being used to indicate that the target data has been read completely. When the second version number is consistent with the third version number and the client receives the first indication information sent by the server, the client stops the cycle process of steps S602-S605, and splices the data segments read according to the read order to obtain the target data.

[0119] Optionally, the client stops the loop and deletes all the read data segments when a second preset condition is met. The second preset condition includes: the second version number is inconsistent with the third version number, where the inconsistency between the second version number and the third version number indicates that the target data has changed; or, an error message sent by the server is received, where the error message is generated by the server in the kth loop when it is determined that the version number in the received data read request is inconsistent with the version number recorded in the KV database, and the error message indicates that the target data has changed.

[0120] The data read method provided by the embodiments of the present application adopts a split type and a cyclic read manner to obtain target data. After the client obtains an idle connection each time, the data read request sent by the client is used to read the target data according to a certain read step, and after receiving the first indication information sent by the server and used to indicate that the target data has been read completely, all the read data segments are spliced to obtain the target data. In this way, the client can read only a certain threshold of data segments each time after obtaining an idle connection each time.

[0121] Through the method provided by the embodiments, on the one hand, when multiple clients access the same KV database, the mutual influence can be guaranteed, the data read request sent by each client can be processed by the KV database, and the problem that the KV database is blocked due to large data accessed by a certain client, the processing request efficiency of the KV database is reduced, and the request processing of other clients is affected can be avoided; on the other hand, the influence between the data read requests corresponding to multiple business threads in the same client can be reduced, and the problem that other requests cannot obtain an idle connection due to that the data accessed by the data read request of a certain business thread is large, the time consumption is too long, and the connection time of the client is occupied for a long time can be avoided.

[0122] The data read method provided by the embodiments of the present application is further explained and described below by taking two specific examples.

[0123] Example 1

[0124] FIG. 8 is a schematic interaction flowchart of a data read method provided by an embodiment of the present application. The method involves the processing logic that a first application in a client reads target data in batches by calling a SDK segmented read command (i.e., cyclically sending data read requests) until the target data is read completely. Specifically, the first application in the client reads target data (Key = string, Value = 123456789009876543211234567890) through a SDK segmented interface: scanStr, and requests a KV database, as shown in FIG. 8. The method includes the following steps S801-S812.

[0125] S801, the first application receives a first operation.

[0126] For example, the first operation can be a click operation on the first application, to instruct the first application to read target data Value = 123456789009876543211234567890 from the KV database.

[0127] S802, the first application sends a data read request to the SDK according to the first operation.

[0128] The data read request is used to request the SDK to read the target data from the KV database in a loop sending data read request mode.

[0129] S803, the SDK obtains an idle connection from N KV database connections according to the data read request.

[0130] S804, the SDK sends a first data read request to the server through the idle connection, the first data read request including a data identifier of the target data, a first read cursor, a read step and a first version number of the target data.

[0131] In this embodiment, the data identifier Key value of the target data is key = string, the first read cursor is cursor = 0, the read step is count = 10, and the first version number is version = 0.

[0132] S805, the server reads data from the KV database according to the first data read request, generates and returns a first read result to the SDK.

[0133] The server determines, according to the data identifier Key value string in the first read command, that the target data is a string 123456789009876543211234567890 corresponding to the string in the KV database, determines, according to the first read cursor cursor = 0 and the read step count = 10, that the first data segment S-1 = 1234567890 to be read this time and the second read cursor cursor = 10 in the next loop read command. It should be noted that in this embodiment, the first version number in the data read request received by the server is 0, so it is determined that the third preset condition is met. In this case, the server directly sends the read result to the client, that is, the server packs the first data segment S-1 = 1234567890, the second read cursor cursor = 10 in the next loop, the second version number version = 1704253321324115 corresponding to the string 123456789009876543211234567890, and the indication information indicating whether the target data is read completely, generates the first read result, and returns it to the SDK.

[0134] S806, the SDK determines whether the first read result contains the first indication information. If not, step S806 is performed.

[0135] The first indication information can be combined with the second read cursor or the second version number, for example, when the read cursor is 0 again, it is determined that the first indication information is included in the first read result. In this embodiment, as described in step S805, the second read cursor cursor = 10 in the read result, so the SDK determines that the first indication information is not included in the read result.

[0136] S807, the SDK determines that the first read result does not contain the first indication information, and reacquires an idle connection from the N KV database connections.

[0137] The SDK determines from the first read result that the first indication information is not included in the first read result, indicating that the target data is not read completely. At this time, an idle connection is reacquired from the N KV database connections to continue reading the next data segment in the target data.

[0138] S808, the SDK sends a second data read request to the server through the idle connection reacquired, the second data read request including a data identifier of the target data, a second read cursor, a read step, and a second version number of the target data.

[0139] In this embodiment, the data identifier Key value key = string of the target data is included in the second read command, the second read cursor cursor = 10, the read step count = 10, and the second version number version = 1704253321324115. That is, at this time, the SDK has updated the first read cursor in the data read request to the second read cursor, the first version number to the second version number, and the like, and similar updates are performed in the subsequent loop process.

[0140] In S809, the server reads data from the KV database according to the second data read request, generates and returns the second read result to the SDK.

[0141] The server determines that the target data is the string 123456789009876543211234567890 corresponding to string in the KV database according to the data identifier Key value string in the second read command, determines the second data segment S-2 = 0987654321 to be read this time and the read cursor cursor = 20 in the next read command according to the second read cursor cursor = 10 and the read step count = 10. Then, the second data segment S-2 = 0987654321, the third read cursor cursor = 20, the third version number version = 1704253321324115 corresponding to the string 123456789009876543211234567890, and the indication information indicating whether the target data is completely read are packaged, the second read result is generated, and the second read result is returned to the SDK.

[0142] In some embodiments, the server compares the second version number in the second read command with the third version number of the target data recorded in the KV database. If they are consistent, it indicates that the target data has changed, for example, has been modified or updated. If they are inconsistent, it indicates that the target data has not changed. At this time, the server directly returns error information to the SDK, and the error information is used to indicate that the target data has changed. In this embodiment, the server compares the second version number version = 1704253321324115 in the second read command with the third version number version = 1704253321324115 recorded in the KV database. The comparison result shows that the version numbers are consistent, and therefore, it is determined that the third preset condition is met, and the error information does not need to be returned to the SDK.

[0143] S810, when the version numbers in the first read result and the second read result are consistent, the SDK determines whether the second read result contains the first indication information, if not, the steps S807-S809 are executed in a loop until the first indication information sent by the server is received, if yes, the step S811 is executed.

[0144] In this embodiment, the version numbers in the first read result and the second read result are consistent, i.e. the second version number version = 1704253321324115 is consistent with the third version number version = 1704253321324115, which indicates that the string has not changed in the process of the cyclic reading by the SDK. At this time, the SDK further determines whether the second read result contains the first indication information, if not, the steps S806-S808 are executed in a loop until the first indication information sent by the database is received, which indicates that the cycle is completed and the target data has been read. If yes, the step S810 is executed.

[0145] In some embodiments, if the SDK receives the error information returned by the server, the SDK determines that the target data has changed, then stops the process of reading data, and reports an error to the first application to indicate the first application to read the target data again.

[0146] S811, the SDK splices all the data segments read according to the reading order to obtain the target data.

[0147] Based on the target data read by the first application through the SDK in this embodiment being the string 123456789009876543211234567890, the reading step of each data reading request being 10, and the version numbers of the target data in each read result being consistent, the SDK can obtain three read results, and the data segments in each read result are respectively: the first data segment S-1 = 1234567890, the second data segment S-2 = 0987654321, and the third data segment S-3 = 1234567890. The SDK splices the three data segments according to the reading order to obtain the target data string 123456789009876543211234567890.

[0148] S812, the SDK returns the target data to the first application.

[0149] It should be noted that in the embodiment, when the first application calls the SDK segmented data reading request to read the target data from the KV database, the data reading request includes the Key, the initial reading cursor, the initial version number and the reading step (i.e. the data size of a single request); the reading result returned by the KV database contains the data segment requested in the data reading request, the next reading cursor and the version number of the target data; the data reading request in the next cycle of the SDK uses the next cycle reading cursor and the version number of the target data returned by the KV database according to the last data reading request, and the cycle is repeated until the first indication information is received, indicating that all data is requested. In the process of reading the target data in the KV database by the application through the SDK, the problem of long reading time, blocking business or KV database caused by large data can be avoided, the latency of the application requesting the KV database is smoothed, and the stable operation of the application and the database is ensured.

[0150] The data reading method provided by the embodiment of the application can be applied in the reading process of string type data. The first application calls the SDK batch data reading request and sends the batch data reading request to the server. The position and length of each reading are controlled through the reading cursor and the reading step, the data segments are read in batches until the reading is completed, and the returned data is spliced. This process can enable the first application to read the string data according to a certain threshold size, and the read data can be trusted based on the version number control, and there is no problem of dirty reading and missing reading. In addition, through the process of segmented cycle reading of the target data, the string does not need to be split into multiple Keys for storage, the reading order needs to be recorded, and additional storage space for multiple Key storage is required, thereby saving costs.

[0151] Example 2

[0152] FIG. 9 is a schematic interaction flowchart of a data reading method provided by an embodiment of the application. The method involves the process of reading the target data by the first application in the client through the SDK segmented reading command (i.e. cyclically sending the data reading request) in batches. The target data is modified by the processing logic of the second application. Specifically, the first application in the client reads the target data (Key = string, Value = 123456789009876543211234567890), and requests the KV database through the SDK segmented interface: scanStr, for example, as shown in FIG. 9. The method includes the following steps S901-S918.

[0153] S901, the first application receives a first operation.

[0154] S902, the first application sends a data reading request to the SDK according to the first operation.

[0155] S903, the SDK acquires an idle connection from the N KV database connections according to the data reading request.

[0156] S904, the SDK sends a first data reading request to the server through the idle connection, the first data reading request comprising a data identifier of the target data, a first reading cursor, a reading step, and a first version number of the target data.

[0157] S905, the server reads data from the KV database according to the first data reading request, generates and returns a first reading result to the SDK.

[0158] The first reading result comprises a data segment corresponding to the target data, a second reading cursor of the target data in the next loop, a second version number of the target data recorded in the KV database, and indication information of whether the target data is read completely.

[0159] S906, the SDK determines whether the first reading result contains the first indication information, if not, step S909 is performed.

[0160] S907, the second application sends a modification command to the server, the modification command being used to instruct the server to modify the target data in the KV database.

[0161] The modification command comprises a data identifier of the target data, i.e., key = string.

[0162] S908, the server modifies the target data in the KV database according to the modification command, and updates the version number of the target data after the modification of the target data is completed.

[0163] S909, the SDK re-acquires an idle connection from the N KV database connections.

[0164] S910, the SDK sends a second data reading request to the server through the re-acquired idle connection, the second data reading request comprising a data identifier of the target data, a second reading cursor, a reading step, and a second version number of the target data.

[0165] S911, the server acquires the second version number of the target data in the second data reading request, and a third version number of the target data recorded in the KV database.

[0166] It should be understood that the second data reading request sent by the client to the server carries the updated second version number of the target data, and the KV database records the third version number generated based on the modification of the target data.

[0167] That is, based on the foregoing steps S907-S908, the server has modified the target data in the KV database according to the modification command sent by the second application, and the KV database adaptively synchronously modifies the version number of the target data, so that the version number of the target data has changed after the present step.

[0168] S912, after the server determines that the second version number of the target data in the second data read request and the third version number of the target data recorded in the KV database are inconsistent, the server returns error information to the SDK, and the error information is used to indicate that the target data has changed.

[0169] It should be understood that in the present example, the server has modified the target data in the KV database based on the modification command of the second application before receiving the second data read request, and synchronously updated the version number of the target data, so that the error information in the present embodiment is generated by the server after receiving the second data read request when it is determined that the second version number of the target data in the second data read request and the third version number of the target data recorded in the KV database are inconsistent, that is, generated in the second loop. When the present method is applied to other embodiments, the error information can be generated by the server in the kth loop when it is determined that the version number in the received data read request and the version number recorded in the database are inconsistent, where k≥2.

[0170] S913, the SDK stops the loop according to the error information, and deletes all the data segments read.

[0171] The SDK stops executing the present data read process according to the error information, and clears all the data segments read.

[0172] S914, the SDK sends an error report to the first application.

[0173] The error report is used to indicate that the target data of the first application has changed, and prompt the first application to re-read the modified target data.

[0174] In the present embodiment, steps S911-S914 are shown in scenario 1 in FIG. 9. Alternatively, as shown in scenario 2 in FIG. 9, steps S911-S914 in the present embodiment can also be:

[0175] S915, the server reads data from the KV database according to the second data read request, generates and returns a second read result to the SDK.

[0176] S916, the SDK obtains the second version number of the target data in the first read result, and the third version number of the target data in the second read result.

[0177] S917, after the SDK determines that the second version number of the target data in the first read result and the third version number of the target data in the second read result are inconsistent, the SDK stops the loop and deletes all the read data segments.

[0178] S918, the SDK sends an error report to the first application.

[0179] The inconsistent version numbers are used to indicate that the target data has been changed.

[0180] The data reading method provided in the embodiment relates to a process in which target data is modified by a second application in the middle of a process in which a first application reads the target data through a batch data reading request of an SDK. The first application calls the batch data reading request of the SDK, sends a request to a server, the server controls the position and length of each reading through a data reading cursor and a reading step length, and judges whether the data is modified through a version number of each request. If the data is modified, an error is returned to notify the first application that the target data has been modified. Through comparison of the data version numbers of the target data, the problem of dirty reading or missing reading is avoided. If the target data is modified by a request of another application in the middle of the reading process, the SDK notifies the current application through an error report and the like, so that the current application avoids reading incorrect data.

[0181] It should be noted that the version number of the record in the KV database involved in the embodiment of the application is a data version number, which is used to compare the new and old data. For example, in a master-slave node data synchronization scenario, the common KV database Redis data does not have a version number, and the master-slave data synchronization is completed through full coverage, that is, the master node sends full data to the slave node, and the slave node unconditionally covers its own data. In order to improve the synchronization efficiency, the incremental message queue is used to send the synchronization data one by one, and if the incremental message queue is full, full data synchronization is triggered, and the slave node still unconditionally covers its own data. However, the KV database involved in the embodiment optimizes the master-slave data synchronization scenario based on the version number. The master node checks the data of the slave node, and if the version number of a piece of data of the slave node is lower than that of the master node, the master node actively pushes the data to the slave node for updating. Conversely, the slave node also checks the data of the master node, and if the version number of a piece of data of the master node is higher than that of the slave node, the slave node actively pulls the data for updating. When the slave node receives the synchronization data, the slave node also checks the synchronization data and its own version number, and only receives the data higher than its own version number. Through the comparison of the version numbers and the checking of the data, the master node can avoid frequent full data synchronization, thereby improving the performance.

[0182] The version number of the relational database MySQL is divided into two kinds. One is a message version number (a sequence number of a binlog), which is used in a scenario of comparing the new and old messages, and is irrelevant to the data itself. The other is a data modification version number, which is only used in a transaction scenario, and is used to judge whether the data can be displayed externally (the transaction is not completed, and the newly written data cannot be displayed externally), and is irrelevant to data synchronization and data new and old judgment. The version number involved in the examples 1 and 2 of the present application is used in a scenario of judging whether the target data is modified, and belongs to the second kind, that is, the data modification version number.

[0183] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0184] The embodiments of the present application also provide a client, which reads data from a KV database through N KV database connections established with a server, N≥1, as shown in FIG. 10, the client comprises a receiving module 1001, an operation module 1002 and a processing module 1003.

[0185] The receiving module 1001 is configured to receive a first operation, the first operation being used to instruct the client to read target data from the KV database.

[0186] The operation module 1002 is configured to cyclically execute the following steps S1-S3 according to the first operation:

[0187] S1, determining an idle connection from the N KV database connections;

[0188] S2, sending a data reading request to the server through the idle connection, the data reading request comprising a first command, a data identifier of the target data, a first reading cursor and a reading step length, the first command being used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the reading cursor and the reading step length;

[0189] S3, receiving a reading result returned by the server, the reading result comprising a data segment corresponding to the target data, a second reading cursor of the target data in the next cycle and indication information of whether the target data is completely read.

[0190] The processing module 1003 is configured to stop the cycle when a first preset condition is met, and splice all the data segments read according to the reading order to obtain the target data.

[0191] Wherein, in each loop, the data identifier of the target data is the same; the read step in the kth loop is determined by the client according to the read result received in the (k-1)th loop, and in the kth loop, the first read cursor is updated to the second read cursor, k≥2.

[0192] The embodiment of the present application further provides a server, which is deployed with a KV database; referring to Fig. 11, the server comprises: a receiving module 1101, an executing module 1102 and a sending module 1103.

[0193] The receiving module 1101 is used for receiving a data read request sent by a client, wherein the data read request comprises a first command, a data identifier of target data, a read cursor and a read step, and the first command is used for instructing the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor and the read step.

[0194] The executing module 1102 is used for executing the first command to obtain the data segment corresponding to the data identifier according to the data read request.

[0195] The sending module 1103 is used for sending a read result to the client.

[0196] The embodiment of the present application further provides a chip, referring to Fig. 12, the chip comprises a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to realize the data read method in each of the above embodiments.

[0197] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the data read method provided in each of the above embodiments.

[0198] The embodiment of the present application further provides a computer program product, which comprises a computer program, and when the computer program is run by an electronic device, the electronic device realizes the data read method provided in each of the above embodiments.

[0199] It should be appreciated that the processor referred to in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0200] It should also be understood that the memory referred to in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0201] In the embodiments provided in the present application, the division of each framework or module is only a logical functional division, and when actually implemented, there can be another division manner, for example, a plurality of frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0202] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can be physically present alone, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0204] In the present application, the reference to "one embodiment" or "some embodiments" means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0205] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A data reading method, characterized in that: Applied to a client, the client reads data from a KV database through N KV database connections established with a server, where N ≥ 1. The method includes: receiving a first operation, where the first operation is used to instruct the client to read target data from the KV database; According to the first operation, the following steps S1 to S3 are executed cyclically: S1, determining an idle connection from the N KV database connections; S2, sending a data read request to the server through the idle connection, the data read request including a first command, a data identifier of target data, a first read cursor, and a read step length, wherein the first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length; S3, receiving a read result returned by the server, wherein the read result includes a data segment corresponding to the target data, a second read cursor of the target data in the next cycle, and an indication of whether all the target data has been read; When the first preset condition is met, the loop is stopped, and all the read data segments are spliced ​​in the reading order to obtain the target data; The read step length in the kth cycle is determined by the client according to the read result received in the k-1th cycle, and in the kth cycle, the first read cursor is updated to the second read cursor, k≥2.

2. The method according to claim 1, characterized in that The read result also includes a second version number of the target data recorded in the KV database.

3. The method according to claim 2, characterized in that The data read request includes a first version number, and the method further includes: In the kth cycle, before sending the data read request to the server through the idle connection, the first version number is updated to the second version number.

4. The method according to claim 2 or 3, characterized in that The method further comprises: A third version number is determined, where the third version number is the second version number of the target data recorded in the KV database and included in the reading result of the k-1th cycle.

5. The method according to claim 4, characterized in that The first preset condition includes: The second version number is consistent with the third version number, and first indication information sent by the server is received, where the first indication information is used to indicate that all target data has been read.

6. The method according to claim 3 or 4, characterized in that The method further comprises: When the second preset condition is met, the loop is stopped and all the read data segments are deleted.

7. The method according to claim 6, characterized in that The second preset condition includes: The second version number is inconsistent with the third version number; or, The reading result includes error information, which is generated by the server in the kth cycle when it determines that the version number in the received data reading request is inconsistent with the version number recorded in the KV database. The error information is used to indicate that the target data has changed.

8. The method according to any one of claims 1 to 7, characterized in that The process of determining the read step length in the k-th cycle includes: Determined based on the response time of the read result received in the k-1th cycle; or, Determined by the number of read results received.

9. The method according to any one of claims 1 to 8, characterized in that The reading step length is smaller than or equal to a first threshold.

10. A data reading method, characterized in that: Applied to a server deployed with a KV database, the method includes: Receive a data read request sent by a client, the data read request including a first command, a data identifier of target data, a read cursor, and a read step, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step; Executing the first command according to the data read request to obtain the data segment corresponding to the data identifier; When a third preset condition is met, a reading result is sent to the client, where the reading result includes a data segment corresponding to the target data.

11. The method according to claim 10, characterized in that The data read request also includes a version number of the target data.

12. The method according to claim 11, characterized in that The third preset condition includes: The version number is 0; or, The version number is not 0, and the version number is consistent with the version number of the target data recorded in the KV database.

13. The method according to claim 11 or 12, characterized in that The method further comprises: When the third preset condition is not met, generating an error message, wherein the error message is used to indicate that the target data has changed; The error information is sent to the client.

14. The method according to any one of claims 10 to 13, characterized in that The step of executing the first command to obtain the data segment corresponding to the data identifier according to the data read request includes: When the read step size is 0, executing the first command according to the data read request to obtain the data segment corresponding to the data identifier with a preset read step size; When the read step size is not 0, the first command is executed according to the data read request to obtain the data segment corresponding to the data identifier with the read step size.

15. A client, characterized in that: The client reads data from the KV database through N KV database connections established with the server, where N≥1. The client includes: A receiving module, configured to receive a first operation, wherein the first operation is configured to instruct the client to read target data from the KV database; The operation module is configured to cyclically execute the following steps S1 to S3 according to the first operation: S1, determining an idle connection from the N KV database connections; S2, sending a data read request to the server through the idle connection, the data read request including a first command, a data identifier of target data, a first read cursor, and a read step length, wherein the first command is used to instruct the server to read the data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length; S3, receiving a read result returned by the server, wherein the read result includes a data segment corresponding to the target data, a second read cursor of the target data in the next cycle, and an indication of whether all the target data has been read; A processing module, configured to stop the loop when a first preset condition is met, and to splice all the read data segments in a reading order to obtain the target data; The read step length in the kth cycle is determined by the client according to the read result received in the k-1th cycle. In the kth cycle, the first read cursor is updated to the second read cursor, and k≥2.

16. A server, characterized in that: The server is deployed with a KV database, and the server includes: a receiving module, configured to receive a data read request sent by a client, wherein the data read request includes a first command, a data identifier of the target data, a read cursor, and a read step length, wherein the first command is used to instruct the server to read a data segment corresponding to the data identifier from the KV database according to the data identifier, the read cursor, and the read step length; an execution module, configured to execute the first command according to the data read request to obtain a data segment corresponding to the data identifier; The sending module is used to send the reading result to the client.

17. A storage system, characterized in that: Includes the client according to claim 15 and the server according to claim 16.

Images