Data access method and device

Abstract

The invention provides a data access method and device. The method comprises the steps that a first access request sent by a first client in a plurality of clients is received, the first client corresponds to a first access protocol, the first access request is used for executing first operation on first data according to a first access protocol request, and the first data is stored in a storage pool or is to be stored in the storage pool; when the first access protocol is inconsistent with a second access protocol corresponding to the storage end, the first access request is converted into a second access request, and the second access request is used for executing a first operation on the first data according to a second access protocol request; sending the second access request to a storage end; wherein the storage end is used for responding to the second access request and executing the first operation for the first data. According to the method, the switching of the storage modes can be realized under the condition that the service code of the client does not need to be transformed.

Description

Technical Field

[0001] The present application relates to the field of cloud technology, and in particular to a data access method and device. Background Art

[0002] With the development of cloud technology, new service models are constantly emerging in cloud storage, such as fully managed NoSQL database services (DynamoDB) and cloud databases (GeminiDB). Compared with DynamoDB, GeminiDB has lower costs. In order to reduce costs and other considerations, users have the need to switch cloud storage service modes.

[0003] Different cloud storage modes may support different access protocols, etc. In related technologies, in order to switch between different cloud storage modes, the service code of the client on the user side is modified so that the client supports the switched cloud storage mode. However, the client service code modification operation is cumbersome and time-consuming, which may affect and cause service interruption. Summary of the invention

[0004] The present application provides a data access method and device, which can realize the switching of storage mode without modifying the business code of the client.

[0005] In a first aspect, a data access method is provided, which is applied to an access device, the access device being connected to a storage end and multiple clients; wherein the storage end performs operations on data stored in a storage pool, and different clients among the multiple clients correspond to different access protocols; the method comprises: receiving a first access request sent by a first client among the multiple clients, the first client corresponds to a first access protocol, the first access request is used to perform a first operation on first data in accordance with the first access protocol request, the first data being stored in the storage pool or to be stored in the storage pool; when the first access protocol and the second access protocol corresponding to the storage end are inconsistent, converting the first access request into a second access request, wherein the second access request is used to perform a first operation on the first data in accordance with the second access protocol request; sending the second access request to the storage end; wherein the storage end is used to respond to the second access request and perform the first operation on the first data.

[0006] When the access protocol of the access request sent by the client is inconsistent with the access protocol of the storage end, the method can convert the access request, and the access protocol of the converted access request is consistent with the access protocol of the storage end. The storage end can respond to the converted access request and perform related operations. In this way, normal access to data can be guaranteed when the access protocol of the client and the access protocol of the storage end are inconsistent, so that the storage mode can be switched without modifying the code of the client. The storage mode can be a cloud storage mode.

[0007] In one possible implementation, the method also includes: receiving a third access request sent by a second client among multiple clients, the second client corresponds to a third access protocol, the third access request is used to perform a second operation on second data in accordance with the third access protocol request, and the second data is stored in a storage pool or is to be stored in a storage pool; when the third access protocol and the second access protocol are consistent, sending the third access request to the storage end; wherein the storage end is used to respond to the third access request and perform the second operation on the second data.

[0008] If the access protocol of the access request sent by the client is consistent with the access protocol of the storage end, there is no need to convert the access request, and the access request sent by the client is directly sent to the storage end, which improves the efficiency of data access.

[0009] In one possible implementation, converting a first access request into a second access request includes: obtaining a primary key from the first access request, wherein data corresponding to the primary key includes first data; carrying the primary key into the second access request; wherein the storage end is used to identify the first data using the primary key in the second access request.

[0010] In this implementation, the primary key in the first access request can be carried into the second access request, so that the storage end can use the primary key to identify the first data in the storage pool, and then perform operations on the first data. Exemplarily, the primary key can be used to identify the data corresponding to the primary key in the storage pool, and the operation is performed on the data corresponding to the primary key, thereby implementing the operation on the first data in the data corresponding to the primary key.

[0011] In a possible implementation, a primary key corresponds to multiple data, wherein the multiple data correspond to multiple attributes one-to-one; obtaining the primary key from a first access request includes: obtaining the primary key and the attributes corresponding to the first data from the first access request; carrying the primary key to a second access request includes: carrying the primary key and the attributes corresponding to the first data to the second access request; wherein the storage end is used to identify the multiple data using the primary key in the second access request, and to identify the first data among the multiple data using the attributes of the first data in the second access request.

[0012] In this implementation, the attributes corresponding to the first data in the first access request can be carried into the second access request, whereby the storage end can use the attributes corresponding to the first data to identify the first data from the data corresponding to the primary key, thereby performing operations only on the first data.

[0013] In a possible implementation, multiple data and multiple attributes are stored in the same column corresponding to the primary key; wherein, in the column, each data in the multiple data is associated with the attribute corresponding to the data. Exemplarily, multiple data and multiple attributes can be stored in the same column via a map data structure, wherein, in the map data structure, the data is associated with the attribute corresponding to the data.

[0014] In this implementation, multiple data and multiple attributes can be stored in the same column corresponding to the primary key, so that data corresponding to multiple attributes can be stored without being able to freely expand the column field.

[0015] In one possible implementation, the first operation is a modification operation; the method further includes: sending a lock indication to the storage end; the lock indication is used to instruct the storage end to block other modification operations on the first data when performing the first operation on the first data.

[0016] This implementation method can ensure that one modification operation is performed on the first data at the same time, thereby avoiding conflicts caused by performing multiple modification operations on the first data at the same time.

[0017] In a second aspect, a data access device is provided, which connects a storage end and multiple clients; wherein the storage end performs operations on data stored in a storage pool, and different clients among the multiple clients correspond to different access protocols; the device includes: a receiving module, used to receive a first access request sent by a first client among the multiple clients, the first client corresponds to a first access protocol, the first access request is used to perform a first operation on first data according to the first access protocol request, and the first data is stored in the storage pool or is to be stored in the storage pool; a conversion module, used to convert the first access request into a second access request when the first access protocol and the second access protocol corresponding to the storage end are inconsistent, wherein the second access request is used to perform a first operation on the first data according to the second access protocol request; a sending module, used to send the second access request to the storage end; wherein the storage end is used to respond to the second access request and perform the first operation on the first data.

[0018] In one possible implementation, the receiving module is also used to: receive a third access request sent by a second client among multiple clients, the second client corresponds to a third access protocol, the third access request is used to perform a second operation on second data in accordance with the third access protocol request, and the second data is stored in a storage pool or is to be stored in a storage pool; the sending module is also used to: when the third access protocol and the second access protocol are consistent, send the third access request to the storage end; wherein the storage end is used to respond to the third access request and perform the second operation on the second data.

[0019] In one possible implementation, the conversion module is used to: obtain a primary key from a first access request, the data corresponding to the primary key includes first data; carry the primary key to a second access request; wherein the storage end is used to identify the first data using the primary key in the second access request.

[0020] In one possible implementation, a primary key corresponds to multiple data, wherein the multiple data correspond one-to-one to the multiple attributes; a conversion module is used to: obtain the attributes corresponding to the primary key and the first data from a first access request; and carry the attributes corresponding to the primary key and the first data to a second access request; wherein the storage end is used to identify the multiple data using the primary key in the second access request, and to identify the first data among the multiple data using the attributes of the first data in the second access request.

[0021] In a possible implementation manner, the multiple data and the multiple attributes are stored in the same column corresponding to the primary key; wherein, in the column, each data of the multiple data is associated with the attribute corresponding to the data.

[0022] In a possible implementation, the first operation is a modification operation; the sending module is further used to: send a lock indication to the storage end; the lock indication is used to instruct the storage end to block other modification operations on the first data when performing the first operation on the first data.

[0023] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method provided in the first aspect.

[0024] In a fourth aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method provided in the second aspect.

[0025] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method provided in the first aspect.

[0026] The beneficial effects of the second to fifth aspects can be referred to the above introduction to the beneficial effects of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0028] Figure 2is a structural diagram of an access device provided in an embodiment of the present application;

[0029] Figure 3 is a flow chart of a data access method provided by an embodiment of the present application;

[0030] Figure 4 It is a structural diagram of a data access device provided in an embodiment of the present application;

[0031] Figure 5 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0033] Figure 7 It is a structural diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The scheme provided by the embodiment of the present application will be described below in conjunction with the accompanying drawings. In the embodiment of the present application, "multiple" means two or more, and "multiple" means two or more. "First", "second", etc. are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.

[0035] To facilitate understanding of the solutions provided by the embodiments of the present application, the technical terms that may be involved in the embodiments of the present application are first introduced.

[0036] Storage pool: refers to a node or cluster with storage resources that can be used to store data. For example, the storage pool can be an infrastructure, and the servers in the infrastructure can be used as storage nodes. Among them, the storage nodes can be used to store data.

[0037] Cloud technology refers to a hosting service that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, networking, processing and sharing.

[0038] Infrastructure: Facilities used to provide resources. An infrastructure can include infrastructure, and each data center includes servers. Among them, servers can store data. Among them, in cloud technology, infrastructure is also called cloud infrastructure.

[0039] Client: A module, device or equipment located on the user side for the user to issue access requests. The client can issue access requests according to the access protocols it supports. The access request is used to perform one or more operations on the relevant data according to the access protocol request.

[0040] Storage end: a module, device, equipment or computing cluster located remotely from the user (such as the cloud), capable of responding to access requests and performing the operations requested by the access request on the relevant data.

[0041] Access protocol: also known as the interaction protocol, is a protocol used for information exchange between the client and the storage end. Generally speaking, the client issues an access request in accordance with the access protocol it supports, and the access request is used to perform one or more operations on the relevant data in accordance with the access protocol request. If the access protocol is also supported by the storage end, the storage end can respond to the access request and perform the operations requested by the access request on the relevant data. The access protocol defines the representation format of the information in the access request, such as the representation format of the primary key, the representation format of the attribute, the format of the condition, etc. Different access protocols have different information representation formats.

[0042] A primary key, also known as a primary keyword, is one or more fields in a data table, and its value is used to uniquely identify a record in the data table. When the data table is a horizontal table, the primary key can be a combination of one or more columns, and its value can uniquely identify a row in the table. For example, the primary key can be composed of a partition key and a cluster key.

[0043] Attribute: It is the property of data, also known as the characteristics or type of data. The same object may have multiple attribute data. For example, different subjects can be different attributes of students. Among them, the grade of a subject is the value of this attribute of the subject.

[0044] Data table: It can be referred to as table for short, and is used to record data. Data tables can be divided into horizontal tables and vertical tables.

[0045] Horizontal table: A data table. In a horizontal table, different data with the same primary key are recorded in the same row. The opposite of a horizontal table is a vertical table. In a vertical table, each row records only one data, that is, different data with the same primary key are recorded in different rows.

[0046] Wide table: It is a horizontal table in the database field that records a large amount of data.

[0047] Map: A data structure that is used to store a collection of key-value pairs. In a map, each element contains a key (key, K) and a value (value, V). The element can be represented as a key-value pair map.<K,V> The key is unique, and the value can be of any data type. The elements in the map have no fixed order, and the key-value pair can be a one-to-one relationship.

[0048] Different cloud storage modes support different access protocols. When switching cloud storage modes, it is necessary to ensure that the access protocol of the access request received by the switched storage end is consistent with the access protocol of the storage end to ensure normal access to data. In related technologies, when switching storage modes, the client is modified to make the access protocol of the access request issued by the client consistent with the access protocol of the switched storage end. Among them, code modification requires professional personnel, and the operation is cumbersome and time-consuming, affecting business continuity.

[0049] In view of this, an embodiment of the present application provides a data access method. In this method, an access device can receive access requests sent by different clients, and convert the access request when the access protocol of the access request is inconsistent with the access protocol of the storage end. The access protocol of the converted access request is consistent with the access protocol of the storage end. The access device sends the converted access request to the storage end, and the storage end can respond to the converted access request and perform the operation requested by the access request. In this way, normal access to data can be guaranteed when the access protocol of the client and the access protocol of the storage end are inconsistent. In other words, the switching of storage mode can be achieved without code modification on the client.

[0050] The access request is used to request to perform related operations on the data according to its access protocol. The access protocol of the storage end can be called the access protocol corresponding to the storage end. The access protocol of the client can be called the access protocol corresponding to the client.

[0051] Next, the data access method provided in the embodiment of the present application is introduced in detail.

[0052] Figure 1 A system architecture that can be used to implement the method is shown. The system architecture includes an access device 110, a storage end 120, a storage pool 130, and multiple clients, such as client 141, client 142, client 143, etc.

[0053] The storage pool 130 has storage resources and can store data. In some embodiments, the storage pool 130 may include at least one storage node, each storage node being used to store data. Exemplarily, the storage node may be a server. Exemplarily, the storage node may be a disk.

[0054] The storage end 120 may be used to perform operations on the storage in the storage pool 130, such as write operations, modify operations, query operations, etc. The storage end 120 is connected to the access device 110 to receive an access request sent by the access device 110. The storage end 120 may respond to the access request and perform the operation requested by the access request on the relevant data in the storage pool 130. Exemplarily, the storage end 120 may be a storage end of GeminiDB.

[0055] The access protocols of different clients in the multiple clients are different. Among them, different clients in the multiple clients may be clients in different storage modes. Exemplarily, client 141 may be a client of DynamoDB, for example, a DynamoDB software development kit (SDK). Client 142 may be a client of GeminiDB. Client 143 may be a client of Amazon Web Services (AWS). Exemplarily, different clients in the multiple clients are located at different user sides.

[0056] The access device 110 is connected to each of the multiple clients to receive an access request sent by each client. The access device 110 can determine whether the access protocol of the received access request is consistent with the access protocol of the storage end 120, and if they are inconsistent, convert the access request, and the access protocol of the converted access request is consistent with the access protocol of the storage end 120. Then, the access device 110 sends the converted access request to the storage end 120.

[0057] The access device 110 may be located at a remote end of the client, for example, the access device 110 is located in the cloud, and the client is located locally. The access device 110 and the client may be connected via a network.

[0058] In some embodiments, the access device 110 is pre-set with the access protocol of each of the multiple clients and the access protocol of the storage end 120. Thus, the access device 110 can understand the access protocol of the client and the access protocol of the storage end 120, thereby determining whether they are consistent, and converting the access protocol of the access request.

[0059] In some embodiments, Figure 1 As shown, the access device 110 and the storage end 120 can be integrated into the management system of the storage pool 130 .

[0060] In some embodiments, Figure 2 As shown, the access device 110 may include a receiving module 111 , a parsing module 112 and an adapting module 113 .

[0061] The receiving module 111 can receive the access request sent by the client, and perform authentication, encoding and decoding, communication protocol analysis and other processing on the access request, so as to obtain the content of the access request after confirming that the access request is legal. The content of the access request can include primary keys, attributes, conditional expressions, modification expressions, projection expressions, etc.

[0062] The parsing module 112 can perform syntax parsing and semantic parsing on the content of the access request to understand the content of the access request. Exemplarily, the parsing module 112 is pre-set with an access protocol corresponding to the client sending the access request. Based on the access protocol, the parsing module 112 can perform syntax parsing and semantic parsing on the content of the access request to understand the content of the access request.

[0063] The adaptation module 113 can express the content of the access request understood by the analysis module 112 in the form defined by the access protocol corresponding to the storage end 120, thereby completing the conversion of the access request, and the converted access request complies with the access protocol corresponding to the storage end 120. In other words, the adaptation module 113 can map the original data format of the access request (the number of data defined by the access protocol corresponding to the client sending the access request) to the data format defined by the access protocol corresponding to the storage end 120, including the mapping of data types, the mapping of primary key table formats, etc.

[0064] The above example introduces the system architecture provided by the embodiment of the present application. Next, taking the implementation in the system architecture as an example, the process of the data access method provided by the embodiment of the present application is introduced. Among them, the method is specifically executed by the access device 110, such as Figure 3 As shown, the method includes the following steps.

[0065] In step 301, the access device 110 receives a first access request sent by a client 141 among multiple clients. The first client 141 corresponds to a first access protocol. The first access request is used to perform a first operation on first data according to the first access protocol request. The first data is stored in the storage pool 130 or is to be stored in the storage pool 130.

[0066] The client 141 may issue a first access request based on the first access protocol, and the first access request is used to request to perform a first operation on the first data. The first data may also be referred to as the access target data of the first access request. The data format of the first access request is the format defined by the first access request. Therefore, it can be said that the first access request is used to request to perform a first operation on the first data according to the first access protocol.

[0067] In some embodiments, the first operation may be a write operation, the first data is a write object, and the first access request is used to request to store the first data in the storage 130 , that is, the first data is to be stored in the storage pool 130 .

[0068] In some embodiments, the first operation may be a query operation or a modification operation. The first data is data stored in the storage pool 130 and is a query object or a modification object of the first operation.

[0069] The client 141 may send a first access request to the access device 110 through the network.

[0070] Step 302: When the first access protocol and the second access protocol corresponding to the storage end 120 are inconsistent, the access device 110 converts the first access request into a second access request, wherein the second access request is used to perform the first operation on the first data according to the second access protocol request.

[0071] The access device 110 may be pre-installed with the first access protocol and the second access protocol, so that the access device 110 can understand the first access protocol and the second access protocol, and thus determine whether they are consistent.

[0072] The access device 110 may perform syntax parsing and semantic parsing on the content of the first access request to understand the content of the access request, such as primary key, attribute, operation type, conditional expression, modification expression or projection expression, etc. Exemplarily, the access device 110 is pre-set with a first access protocol. Based on the first access protocol, the access device 110 performs syntax parsing and semantic parsing on the content of the first access request to understand the content of the first access request.

[0073] In some embodiments, the first access request carries a primary key, and the data corresponding to the primary key includes the first data. In step 302, the access device 110 may obtain the primary key from the first access request and carry the primary key to the second access request. The access device 110 may perform syntax and semantic understanding on the primary key in the first access request, and package or express the understood primary key according to the second access protocol, and carry the primary key packaged or expressed according to the second access protocol to the second access request.

[0074] In an example of this embodiment, the primary key may correspond to multiple data, wherein the multiple data correspond to multiple attributes one by one. The multiple data include first data, and the first access request carries the attributes corresponding to the first data. In step 302, the access device 110 may obtain the attributes corresponding to the primary key and the first data from the first access request, and then carry the attributes corresponding to the primary key and the first data to the second access request. The access device 110 may perform grammatical and semantic understanding of the attributes corresponding to the primary key and the first data in the first access request, and package or express the understood attributes corresponding to the primary key and the first data according to the second access protocol, and carry the attributes corresponding to the primary key and the first data packaged or expressed according to the second access protocol to the second access request.

[0075] In some embodiments, the access device 110 may use the parser generator ANTLR4 to implement semantics and semantic parsing.

[0076] In an illustrative example of this embodiment, taking parsing conditional expressions and modification expressions as an example, UpdateItem can be used to parse conditional expressions and modification expressions, which describes how to modify an Item.

[0077] In an illustrative example of this embodiment, conditional expressions are used in several different places, such as FilterExpression in scans, KeyConditionExpression in queries, and ConditionExpression in updates. Because these expressions support parentheses and can be nested, for example, (NOT (a = b OR c = d)) and e = f.

[0078] In an illustrative example of this embodiment, the projection expression is a list of document paths. These paths are also used in the conditional expression above, the modification expression, which can be a syntax like ab[3].

[0079] After completing the parsing of the content of the first access request, the access device 110 can package the content of the first access request understood by the access device 110 according to the second access protocol to obtain a second access request. Specifically, the access device 110 can express the understood content of the first access request in the form defined by the second access protocol to obtain the second access request. In other words, the access device 110 can map the original data format (the number of data defined by the first access protocol) of sending the first access request to the data format defined by the first access protocol, including the mapping of data types, the mapping of primary key table formats, the mapping of operation types, the mapping of attributes, the mapping of conditional expressions, the mapping of modification expressions, the mapping of projection expressions, etc.

[0080] The second access request is obtained based on the second access protocol and the first access request. The access protocol of the second access request is the second access protocol, the target data is the first data, and the target operation is the first operation, that is, the second access request can perform the first operation on the first data according to the second access protocol request.

[0081] In step 303 , the access device 110 sends a second access request to the storage end 120 ; wherein the storage end 120 is used to respond to the second access request and perform a first operation on the first data.

[0082] The storage end 120 receives the second access request sent by the access device 110. The access protocol of the second access request is the second access protocol, that is, the access protocol of the second access request is consistent with the access protocol corresponding to the storage end 120. Therefore, the storage end 120 can respond to the second access request and perform the target operation (that is, the first operation) of the second access request on the target data (that is, the first data) of the second access request.

[0083] In some embodiments, as described above, the second access request carries a primary key, and the data corresponding to the primary key includes the first data. The storage end 120 can use the primary key of the second access request to identify the first data, and then perform the first operation on the first data.

[0084] In an example of this embodiment, the storage end 120 can use the primary key of the second access request to identify the data corresponding to the primary key in the storage pool 130, and perform a first operation on the data corresponding to the primary key. The data corresponding to the primary key includes the first data, and by performing the first operation on the data corresponding to the primary key, the first operation is performed on the first data. Generally speaking, in a data table, the data corresponding to the primary key is the entire row of data where the primary key is located, and the first operation can be performed on the entire row of data.

[0085] In one example, the first operation may be a query operation, the storage end 120 may obtain the entire row of data from the storage pool 130, and send the entire row of data to the access device 110, and the access device 110 sends the entire row of data to the client 141. After receiving the entire row of data from the storage end 120, the access device 110 may package or represent the entire row of data according to the first access protocol, and then send the entire row of data packaged or represented according to the first access protocol to the client 141. In this way, the first data may be sent to the client 141.

[0086] In an example, the first operation may be a modification operation, and the storage end 120 may modify the entire row of data to modify the first data.

[0087] In one example, the first operation may be a delete operation, and the storage end 120 may delete the entire row of data to delete the first data.

[0088] In one example, the first operation may be a write operation. If the data table of the storage pool 130 does not have a row corresponding to the primary key, a new row corresponding to the primary key is created in the data table, and the first data is stored in the new row. If the data table already has a row corresponding to the primary key, the first data is used to replace the original data in the row.

[0089] In another example of the above embodiment, as described above, the second access request carries a primary key and an attribute of the first data, wherein the primary key corresponds to a plurality of data, and the plurality of data includes the first data. The storage end 120 can use the primary key to identify the plurality of data in the storage pool 130, and then use the attribute of the first data to identify the first data in the plurality of data, and then perform the first operation on the first data.

[0090] In one example, the first operation may be a query operation, and the second access request carries a projection expression, wherein the projection expression includes a primary key and an attribute of the first data. The storage end 120 may use the projection expression to query the first data in the storage pool 130, and send the first data to the access device 110 through the access device 110.

[0091] In one example, the first operation may be a write operation, and the second access request carries a conditional expression, wherein the conditional expression includes a primary key and an attribute of the first data. The storage end 120 may write the first data to the storage pool 130 using the conditional expression.

[0092] In one example, the first operation may be a modification operation, and the second access request carries a conditional expression, wherein the conditional expression includes a primary key and an attribute of the first data. If the first data is new data corresponding to the primary key and the data, the storage end 120 may modify the old data corresponding to the primary key and the attribute to the first data using the conditional expression. The storage end 120 may return the old data to the client 141 through the access device 110. If the first data is old data corresponding to the primary key and the data, the storage end 120 may modify the first data in the storage pool 130 to new data using the conditional expression. The storage end 120 may return the new data to the client 141 through the access device 110.

[0093] In one example, the first operation may be a modification operation, and the second access request carries a conditional expression, wherein the conditional expression includes a primary key and an attribute of the first data. The storage end 120 may delete the first data in the storage pool 130 using the conditional expression, and return the deletion result to the client 141 through the access device 110.

[0094] In one example, multiple data corresponding to a primary key and multiple attributes corresponding to the multiple data are stored in the same column corresponding to the primary key; wherein, in the column, each data in the multiple data is associated with the attribute corresponding to the data. wherein, the multiple data corresponding to the multiple attributes specifically means that the multiple data and the multiple attributes are in one-to-one correspondence.

[0095] In this example, the client 141 may be a DynamoDB client, and the storage end 120 may be a GeminiDB storage end.

[0096] DynamoDB is a model-free database whose columns are extensible. When creating a table, you only need to specify the primary key, and data with different attributes are stored in different columns. Taking the student table that stores student-related data as an example, the student table in DynamoDB can be shown in Table 1.

[0097] Table 1

[0098] Student ID (primary key) Language (attribute A1) Mathematics (Attribute A2) Physical (Attribute A3) 1xxxxxx1 86 95 80 1xxxxxx2 80 96 86 1xxxxxx3 90 85 75 1xxxxxx4 75 98 90

[0099] Among them, the primary key of DynamoDB can be mapped to the primary key field of the data table of GeminiDB (such as a wide table), that is, the primary key of DynamoDB can be recorded in the column corresponding to the primary key field of the data table of GeminiDB. Multiple attributes of DynamoDB can be mapped to the same column of the data table of GeminiDB. Exemplarily, the map data structure can be used to store multiple attributes and the data corresponding to the multiple attributes. Among them, the key in the map represents the attribute, and the value in the map represents the data. Exemplarily, the key in the map can be expressed as 8 bits (Unicode Transformation Format, UTF-8), and the value can be expressed as a byte, that is, the value can be the binary after the data is serialized. In other words, each attribute and the data corresponding to the attribute can be expressed as: map<UTF-8,Bytes> The data in Table 1 is stored in the data table of GeminiDB as shown in Table 2.

[0100] Table 2

[0101]

[0102]

[0103] The storage end 120 can use the primary key in the second access request to identify the map where the first data is located. Then, use the attribute of the first data in the second access request to identify the first data in the map. For example, if the primary key is 1xxxxxx1 and the attribute is language, use 1xxxxxx1 to identify that the map where the first data is located is {<language, 86>, <mathematics, 95>, <physics, 80>}, and then use language to identify that the first data is 86.

[0104] In some embodiments, the first operation may be a modification operation, which is used to modify the first data in the resource pool 130. When the access device 110 sends the second access request to the storage end 120, it may send a locking indication to the storage end 120. The locking indication is used to instruct the storage end 120 to block other modification operations on the first data when performing the first operation on the first data. The storage end 120 may respond to the locking indication and block other modification operations on the first data when performing the first operation on the first data, so as to ensure the atomicity of concurrent modifications and avoid conflicts caused by the storage end 120 performing multiple modification operations on the first data at the same time. Among them, other modification operations refer to modification operations other than the first operation. Exemplarily, the storage end 120 may apply a row lock to the row corresponding to the primary key to block other modification operations on the first data. Among them, the data corresponding to the primary key includes the first data.

[0105] In an example of this embodiment, the code for the storage end 120 to apply a row lock to a row is as follows:

[0106]

[0107] In some embodiments, the access device 110 may also receive a third access request sent by a client 142 among the multiple clients, the third client corresponds to the third access protocol, and the third access request is used to request to perform a second operation on the second data according to the third access protocol, and the second data is stored in the storage pool 130 or is to be stored in the storage pool 130. When the third access protocol and the second access protocol are consistent, the access device 110 sends the third access request to the storage end 120; wherein the storage end 120 is used to respond to the third access request and perform the second operation on the second data.

[0108] If the access protocol of the access request received by the access device 110 from the client is consistent with the access protocol corresponding to the storage end 120, the access device 110 can convert the access request and send the received access request to the storage end 120. The access protocol of the access request is consistent with the access protocol corresponding to the storage end 120, and the storage end 120 can parse and respond to the access request and perform the operation requested by the access request on the target data of the access request.

[0109] In summary, the data access method provided by the embodiment of the present application can ensure normal data access when the access protocol of the client and the access protocol of the storage end are inconsistent, thereby realizing the switching of the storage mode without the need to modify the code of the client.

[0110] Based on the above description, an embodiment of the present application provides a data access device 400. The device connects a storage end and multiple clients; wherein the storage end performs operations on data stored in a storage pool, and different clients among the multiple clients correspond to different access protocols.

[0111] like Figure 4 As shown, the device 400 includes:

[0112] A receiving module 410 is configured to receive a first access request sent by a first client among multiple clients, the first client corresponds to a first access protocol, the first access request is used to perform a first operation on first data according to the first access protocol request, and the first data is stored in a storage pool or is to be stored in the storage pool;

[0113] A conversion module 420, configured to convert a first access request into a second access request when the first access protocol and a second access protocol corresponding to the storage end are inconsistent, wherein the second access request is used to perform a first operation on the first data according to the second access protocol request;

[0114] The sending module 430 is used to send the second access request to the storage end; wherein the storage end is used to respond to the second access request and perform the first operation on the first data.

[0115] In some embodiments, the receiving module 410 is also used to: receive a third access request sent by a second client among multiple clients, the second client corresponds to a third access protocol, the third access request is used to perform a second operation on the second data in accordance with the third access protocol request, and the second data is stored in the storage pool or is to be stored in the storage pool; the sending module 430 is also used to: when the third access protocol and the second access protocol are consistent, send the third access request to the storage end; wherein the storage end is used to respond to the third access request and perform the second operation on the second data.

[0116] In some embodiments, the conversion module 420 is used to: obtain a primary key from a first access request, the data corresponding to the primary key includes first data; carry the primary key to a second access request; wherein the storage end is used to identify the first data using the primary key in the second access request.

[0117] In an example of this embodiment, a primary key corresponds to multiple data, wherein the multiple data correspond to multiple attributes one-to-one; the conversion module 420 is used to: obtain the attributes corresponding to the primary key and the first data from the first access request: carry the attributes corresponding to the primary key and the first data to the second access request; wherein the storage end is used to identify the multiple data using the primary key in the second access request, and to identify the first data among the multiple data using the attributes of the first data in the second access request.

[0118] In one example of this example, multiple data and multiple attributes are stored in the same column corresponding to the primary key; wherein, in the column, each data of the multiple data is associated with the attribute corresponding to the data.

[0119] In some embodiments, the first operation is a modification operation; the sending module is further used to: send a lock indication to the storage end; the lock indication is used to instruct the storage end to block other modification operations on the first data when performing the first operation on the first data.

[0120] Among them, the receiving module 410, the conversion module 420 and the sending module 430 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the receiving module 410 is introduced below by taking the receiving module 410 as an example. Similarly, the implementation of the conversion module 420 and the sending module 430 can refer to the implementation of the receiving module 410.

[0121] As an example of a software functional unit, the receiving module 410 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the receiving module 410 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone AZ or in different AZs, and each AZ includes a data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.

[0122] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same VPC or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0123] As an example of a hardware functional unit, the receiving module 410 may include at least one computing device, such as a server, etc. Alternatively, the receiving module 410 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0124] The multiple computing devices included in the receiving module 410 can be distributed in the same region or in different regions. The multiple computing devices included in the receiving module 410 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the receiving module 410 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0125] It should be noted that, in other embodiments, the receiving module 410 may be used to execute Figure 3 In any step of the method shown, the conversion module 420 can be used to perform Figure 3 In any step of the method shown, the sending module 430 can be used to perform Figure 3 The steps implemented by the receiving module 410, the conversion module 420 and the sending module 430 can be specified as needed, and the steps implemented by the receiving module 410, the conversion module 420 and the sending module 430 can be specified as needed. Figure 3 The different steps in the method shown are used to implement the overall functions of the apparatus 400 .

[0126] The present application also provides a computing device 500. Figure 5 As shown, computing device 500 includes: bus 502, processor 504, memory 506 and communication interface 508. Processor 504, memory 506 and communication interface 508 communicate through bus 502. Computing device 500 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 500.

[0127] The bus 502 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus 502 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 502 may include a path for transmitting information between various components of the computing device 500 (eg, the memory 506, the processor 504, and the communication interface 508).

[0128] The processor 504 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0129] The memory 506 may include a volatile memory, such as a random access memory (RAM). The memory 506 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0130] The memory 506 stores executable program codes, and the processor 504 executes the executable program codes to respectively implement the functions of the aforementioned receiving module 410, the conversion module 420, and the sending module 430, thereby implementing Figure 3 That is, the memory 506 stores the method for executing Figure 3 Instructions for the method shown.

[0131] The communication interface 508 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 500 and other devices or a communication network.

[0132] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0133] like Figure 6 As shown, the computing device cluster includes at least one computing device 500. The memory 506 in one or more computing devices 500 in the computing device cluster may store the same Figure 3 Instructions for the method shown.

[0134] In some possible implementations, the memory 506 of one or more computing devices 500 in the computing device cluster may also store a program for executing Figure 3 In other words, the combination of one or more computing devices 500 can jointly execute instructions for executing Figure 3 Instructions for the method shown.

[0135] It should be noted that the memory 506 in different computing devices 500 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the apparatus 400. That is, the instructions stored in the memory 506 in different computing devices 500 may implement the functions of one or more modules among the receiving module 410, the conversion module 420 and the sending module 430.

[0136] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 7 A possible implementation is shown. Figure 7 As shown, two computing devices 500A and 500B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 506 in the computing device 500A stores instructions for executing the functions of the receiving module 410. At the same time, the memory 506 in the computing device 500B stores instructions for executing the functions of the conversion module 420 and the sending module 430.

[0137] It should be understood that Figure 7 The functions of the computing device 500A shown in FIG. 5A may also be completed by multiple computing devices 500. Similarly, the functions of the computing device 500B may also be completed by multiple computing devices 500.

[0138] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 6 and Figure 7 The difference is that the memory 506 in one or more computing devices 500 in the computing device cluster may store the same memory for executing Figure 3 Instructions for the method shown.

[0139] In some possible implementations, the memory 506 of one or more computing devices 500 in the computing device cluster may also store a program for executing Figure 3 In other words, the combination of one or more computing devices 500 can jointly execute instructions for executing Figure 3 Instructions for the method shown.

[0140] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes Figure 3 The method shown.

[0141] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a host migration device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute Figure 3 The method shown.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data access method, characterized in that: The method is applied to an access device, wherein the access device is connected to a storage end and multiple clients; wherein the storage end performs operations on data stored in a storage pool, and different clients among the multiple clients correspond to different access protocols; the method comprises: receiving a first access request sent by a first client among the multiple clients, the first client corresponding to a first access protocol, the first access request being used to perform a first operation on first data according to the first access protocol request, the first data being stored in the storage pool or to be stored in the storage pool; When the first access protocol and the second access protocol corresponding to the storage end are inconsistent, converting the first access request into a second access request, wherein the second access request is used to perform the first operation on the first data according to the second access protocol request; The second access request is sent to the storage end; wherein the storage end is used to respond to the second access request and perform the first operation on the first data.

2. The method according to claim 1, characterized in that The method further comprises: receiving a third access request sent by a second client among the multiple clients, the second client corresponding to the third access protocol, the third access request being used to perform a second operation on second data according to the third access protocol request, the second data being stored in the storage pool or to be stored in the storage pool; When the third access protocol is consistent with the second access protocol, the third access request is sent to the storage end; wherein the storage end is used to respond to the third access request and perform the second operation on the second data.

3. The method according to claim 1 or 2, characterized in that: The converting the first access request into a second access request comprises: Acquire a primary key from the first access request, where the data corresponding to the primary key includes the first data; The primary key is carried into the second access request; wherein the storage end is used to identify the first data using the primary key in the second access request.

4. The method according to claim 3, characterized in that The primary key corresponds to a plurality of data, wherein the plurality of data corresponds to a plurality of attributes one by one; The acquiring the primary key from the first access request includes: acquiring the primary key and the attribute corresponding to the first data from the first access request; Carrying the primary key into the second access request includes: carrying the primary key and the attributes corresponding to the first data into the second access request; wherein the storage end is used to identify the multiple data using the primary key in the second access request, and to identify the first data among the multiple data using the attributes of the first data in the second access request.

5. The method according to claim 4, characterized in that The plurality of data and the plurality of attributes are stored in the same column corresponding to the primary key; wherein, in the column, each data of the plurality of data is associated with the attribute corresponding to the data.

6. The method according to any one of claims 1 to 5, characterized in that The first operation is a modification operation; the method further comprises: A locking indication is sent to the storage end; the locking indication is used to instruct the storage end to block other modification operations on the first data when performing the first operation on the first data.

7. A data access device, characterized in that: The device is connected to a storage end and multiple clients; wherein the storage end performs operations on data stored in a storage pool, and different clients among the multiple clients correspond to different access protocols; the device includes: a receiving module, configured to receive a first access request sent by a first client among the multiple clients, the first client corresponding to a first access protocol, the first access request being used to perform a first operation on first data according to the first access protocol request, the first data being stored in the storage pool or to be stored in the storage pool; a conversion module, configured to convert the first access request into a second access request when the first access protocol and a second access protocol corresponding to the storage end are inconsistent, wherein the second access request is used to perform the first operation on the first data according to the second access protocol request; A sending module is used to send the second access request to the storage end; wherein the storage end is used to respond to the second access request and perform the first operation on the first data.

8. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by a computer device cluster, the computer device cluster executes the method according to any one of claims 1 to 6.

Images