Data storage method and device, database access method and device, electronic equipment and medium

Abstract

The invention relates to a data storage method and device, a database access method and device, electronic equipment and a medium, the method can be used for a distributed database, and the method comprises the steps that a data storage request is received, and the data storage request carries a request user identifier and to-be-stored data; determining a corresponding target user data scale label according to the request user identifier; determining a target packet routing algorithm corresponding to the target user data scale label according to the data routing packet configuration information; the data routing grouping configuration information is used for dividing user data of different scales into data tables of different storage nodes of a database, so that data magnitude loads among the storage nodes are balanced; and according to the target packet routing algorithm, grouping and forwarding the to-be-stored data to the corresponding storage nodes, and constructing a storage mapping relationship between the to-be-stored data and the stored data table. Uniform distributed storage under different user data volume levels is realized, and the method is adapted to the situation of dynamic change of user data volume.

Description

Technical Field

[0001] The present disclosure relates to the field of distributed storage technology, and in particular to a method, device, electronic device and medium for data storage and database access. Background Art

[0002] With the exponential increase in the amount of data in various applications and the emergence of high-concurrency access scenarios, the requirements for database capacity and access performance are becoming higher and higher. As a result, distributed storage technology has emerged. The basic idea of ​​distributed storage is to disperse the data in the original centralized database to multiple data storage nodes connected by the network to obtain larger storage capacity and higher concurrent access.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following technical problems in the related technology: when storing data based on a distributed database, most of the related technologies use user differences to divide the data into data tables of different storage nodes for storage. However, this division process will make the storage capacity of some data tables very large, and the storage capacity of some data tables small, resulting in serious data skew. In the process of application access to the database, the problem of accessing a certain storage node in the hot data center will also occur; in the scenario where the user data volume changes dynamically, this division method cannot be adapted to the situation where the user data volume changes dynamically. Therefore, the storage logic in the related technology has an adverse effect on the performance of the distributed database and the stability of the storage architecture when facing the situation where the magnitude of different user data is greatly different. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a method, device, electronic device and medium for data storage and database access.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for data storage. The data storage method includes: receiving a data storage request, the data storage request carries a requesting user identifier and data to be stored; determining a corresponding target user data scale label according to the requesting user identifier; determining a target group routing algorithm corresponding to the target user data scale label according to data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of a database, so that the data level load between each storage node is balanced; according to the target group routing algorithm, the data to be stored is grouped and forwarded to the corresponding storage node, and a storage mapping relationship between the data to be stored and the stored data table is constructed.

[0006] According to an embodiment of the present disclosure, the data routing group configuration information includes: a correspondence between a user data scale label and a group routing algorithm; the determining the target group routing algorithm corresponding to the target user data scale label according to the data routing group configuration information includes: determining the target group routing algorithm corresponding to the target user data scale label according to the correspondence; in the case where the target user data scale label is a user label of the first level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored, so that the data to be stored is divided into a first number of data tables; in the case where the target user data scale label is a user label of the second level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored, obtaining a first logical grouping result; performing a second routing grouping operation on the first logical grouping result, so that the data to be stored is divided into a second number of data tables; wherein the second level scale is larger than the first level scale, and the second number is larger than the first number.

[0007] According to an embodiment of the present disclosure, the first routing grouping operation includes: a hash modulus operation; the second routing grouping operation includes: performing a grouping operation according to a preset time period; and the attribute information includes: a requesting user identifier.

[0008] According to an embodiment of the present disclosure, according to the above-mentioned request user identifier, determining the corresponding target user data scale label includes: based on the above-mentioned request user identifier, matching in a pre-configured user label configuration table to obtain a matching user data scale label; the above-mentioned user label configuration table includes: user identifier, user data scale label and label time; the above-mentioned label time is used to indicate the effective time of the user data scale label; according to the matching label time corresponding to the above-mentioned matching user data scale label, determining the target user data scale label corresponding to the receiving time of the above-mentioned data storage request.

[0009] According to an embodiment of the present disclosure, the above-mentioned tag time is represented by the starting moment of the user data scale tag, which is used to indicate that the time period from the previous starting moment to the next starting moment is valid. Wherein, according to the matching tag time corresponding to the above-mentioned matching user data scale tag, the target user data scale tag corresponding to the receiving moment of the above-mentioned data storage request is determined, including: when the above-mentioned matching tag time contains multiple matching starting moments and they are all earlier than the above-mentioned receiving moment, the matching user data scale tag corresponding to the latest matching starting moment among the above-mentioned multiple matching starting moments is determined as the above-mentioned target user data scale tag; when the above-mentioned matching tag time contains multiple matching starting moments and the above-mentioned receiving moment falls into the time period between the first matching starting moment and the second matching starting moment, the matching user data scale tag corresponding to the above-mentioned first matching starting moment is determined as the above-mentioned target user data scale tag.

[0010] According to an embodiment of the present disclosure, the above-mentioned tag time is represented by the duration information of the user data scale tag; wherein, according to the matching tag time corresponding to the above-mentioned matching user data scale tag, the target user data scale tag corresponding to the receiving moment of the above-mentioned data storage request is determined, including: when the above-mentioned matching tag time contains one or more matching duration information, determining the target matching duration information into which the above-mentioned receiving moment falls; and determining the matching user data scale tag corresponding to the above-mentioned target matching duration information as the above-mentioned target user data scale tag.

[0011] According to an embodiment of the present disclosure, the above-mentioned data storage method also includes: generating scale label information within the above-mentioned statistical period for the above-mentioned target user based on the stored data of the target user within the statistical period and at least one of the database access performance; and generating or updating the user data scale label in the above-mentioned user label configuration table based on the above-mentioned scale label information.

[0012] According to an embodiment of the present disclosure, the above-mentioned data storage method is applied to a storage service end including a database middleware or a storage demand end including a database middleware. Among them, according to the above-mentioned target group routing algorithm, the above-mentioned data to be stored is grouped and forwarded to the corresponding storage node, and a storage mapping relationship between the above-mentioned data to be stored and the stored data table is constructed, including: executing the above-mentioned target group routing algorithm on the attribute information of the above-mentioned data to be stored, and dividing the above-mentioned data to be stored into the corresponding data tables of different storage nodes of the above-mentioned database; constructing the association relationship between the data to be stored and the stored data table; according to the request user identifier of the above-mentioned data to be stored, the target user data scale label and the corresponding label time, the above-mentioned association relationship is associated and expanded to obtain a multi-dimensional storage mapping relationship; the above-mentioned storage mapping relationship includes: the association relationship between the user identifier, the user data scale label, the label time and the stored data table.

[0013] In the second aspect, the embodiments of the present disclosure provide a method for data access. The method is applied to a storage demand end including a database middleware, and includes: receiving a data access request, the data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; according to the access user identifier, determining the corresponding access user data scale label; according to a pre-constructed storage mapping relationship, querying a target data table corresponding to the access user identifier and the access user data scale label; forwarding the data operation instruction for the data to be accessed in the target data table to the corresponding storage node.

[0014] In the third aspect, the embodiments of the present disclosure provide a method for data access. The method is applied to a storage service end including a database middleware, and includes: receiving a data access request, the data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; according to the access user identifier, determining the corresponding access user data scale label; according to a pre-constructed storage mapping relationship, querying a target data table corresponding to the access user identifier and the access user data scale label; executing the data operation instruction for the data to be accessed in the target data table to obtain a data operation result.

[0015] According to an embodiment of the present disclosure, in the data access method provided in the second aspect or the third aspect, the data access request also carries: a search period for the data to be accessed; the storage mapping relationship includes: an association relationship between the user identifier, the user data scale label, the label time and the stored data table; the label time is used to indicate the effective time of the user data scale label. According to the pre-constructed mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale label includes: filtering the target label time that matches the search period in the label time of the mapping storage relationship; querying the target data table corresponding to the access user identifier and the access user data scale label within the limited time period of the target label time.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a device for data storage. The above-mentioned device is a storage demand end including a database middleware or a storage service end including a database middleware, and includes: a storage request receiving module, a storage label determination module, a routing algorithm determination module and a group routing module. The above-mentioned storage request receiving module is used to receive a data storage request, and the above-mentioned data storage request carries a requesting user identifier and data to be stored. The above-mentioned storage label determination module is used to determine the corresponding target user data scale label according to the above-mentioned requesting user identifier. The above-mentioned routing algorithm determination module is used to determine the target group routing algorithm corresponding to the above-mentioned target user data scale label according to the data routing group configuration information; the above-mentioned data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data level load between each storage node is balanced. The above-mentioned group routing module is used to group and forward the above-mentioned data to be stored to the corresponding storage node according to the above-mentioned target group routing algorithm, and construct a storage mapping relationship between the above-mentioned data to be stored and the stored data table.

[0017] In a fifth aspect, an embodiment of the present disclosure provides a device for data access. The above-mentioned device is a storage demand end including a database middleware, and includes: a first access request receiving module, a first access label determination module, a first storage table query module and a forwarding module. The above-mentioned first access request receiving module is used to receive a data access request, and the above-mentioned data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the above-mentioned data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction. The above-mentioned first access label determination module is used to determine the corresponding access user data scale label according to the above-mentioned access user identifier. The above-mentioned first storage table query module is used to query the target data table corresponding to the above-mentioned access user identifier and the above-mentioned access user data scale label according to a pre-constructed storage mapping relationship. The above-mentioned forwarding module is used to forward the data operation instruction for the data to be accessed in the target data table to the corresponding storage node.

[0018] In a sixth aspect, an embodiment of the present disclosure provides a device for data access. The above-mentioned device is a storage service end including a database middleware, and includes: a second access request receiving module, a second access label determination module, a second storage table query module and a data operation module. The above-mentioned second access request receiving module is used to receive a data access request, and the above-mentioned data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the above-mentioned data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction. The above-mentioned second access label determination module is used to determine the corresponding access user data scale label according to the above-mentioned access user identifier. The above-mentioned second storage table query module is used to query the target data table corresponding to the above-mentioned access user identifier and the above-mentioned access user data scale label according to a pre-constructed storage mapping relationship. The above-mentioned data operation module is used to execute the above-mentioned data operation instruction for the data to be accessed in the above-mentioned target data table to obtain a data operation result.

[0019] In a seventh aspect, an embodiment of the present disclosure provides an electronic device. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the above-mentioned data storage method or data access method when executing the program stored in the memory.

[0020] In an eighth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for storing data or the method for accessing data as described above is implemented.

[0021] The above technical solution provided by the embodiments of the present disclosure has at least some or all of the following advantages:

[0022] Since different users have their own user data scale labels, different group routing algorithms are configured for different user data scale labels in the data routing group configuration information. After receiving a data storage request, the corresponding target user data scale label is determined according to the requesting user identifier, and the target group routing algorithm corresponding to the above target user data scale label is determined according to the data routing group configuration information; accordingly, the corresponding target group routing algorithm is executed, so that user data of different scales can achieve data volume load balancing between each storage node in the process of dividing the data tables corresponding to each storage node of the database. On the one hand, uniform group storage under different user data volume levels is achieved to avoid large differences in the quantity level of data storage between different storage nodes, effectively solve the problem of data skew, and also effectively avoid the impact of access concentration caused by the concentration of hot data on a certain storage node on the stability of the database; on the other hand, when the user's data volume changes dynamically, it will also reflect the changes in the corresponding user data scale label and the corresponding group routing algorithm, so as to dynamically adapt to the situation where the user data volume changes and perform corresponding balanced storage and access, thereby realizing dynamic adaptive storage and access. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0025] Figure 1A A system architecture of a data storage method and a database access method applicable to an embodiment of the present disclosure is schematically shown;

[0026] Figure 1B Another system architecture of the data storage method and database access method applicable to the embodiment of the present disclosure is schematically shown;

[0027] Figure 2 A flowchart of a data storage method according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 3 The detailed implementation flow chart of step S220 according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 4The detailed implementation flow chart of step S320 according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 5 The detailed implementation flow chart of step S240 according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 6 A flowchart of a method for data access applied to a storage demand side according to an embodiment of the present disclosure is schematically shown;

[0032] Figure 7 A flowchart of a method for data access applied to a storage service client according to an embodiment of the present disclosure is schematically shown;

[0033] Figure 8 The structure block diagram of a data storage device according to an embodiment of the present disclosure is schematically shown;

[0034] Fig. 9 The structure block diagram schematically shows a data access device according to an embodiment of the present disclosure;

[0035] Fig.10 The structure block diagram schematically shows a data access device according to another embodiment of the present disclosure;

[0036] Fig.11 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0038] Figure 1A A system architecture of a data storage method and a database access method applicable to an embodiment of the present disclosure is schematically shown; Figure 1B Another system architecture of the data storage method and database access method applicable to the embodiments of the present disclosure is schematically shown.

[0039] Reference Figure 1A and Figure 1B As shown, the system architecture of the data storage method and database access method applicable to the embodiments of the present disclosure includes: a storage demand end 110 and a storage service end 120.

[0040] The storage demand end 110 may be various devices with data storage requirements, such as terminal devices with various applications 111 installed or servers with data storage requirements (such as servers that provide data computing service support for the applications 111).

[0041] The above-mentioned applications 111 include but are not limited to: shopping applications, web browser applications, search applications, short video applications, video playback applications, image editing applications, instant messaging tools, email clients, social platform software, etc.

[0042] The storage service end 120 is used to provide data storage services for the storage demand end 110. In some embodiments, the storage service end 120 may have a database or be able to access a database.

[0043] Figure 1A Example system architecture 100a and Figure 1B The differences between the exemplary system architecture 100b are: differences in the proxy mode of the database middleware, Figure 1A The example is the server proxy mode. Figure 1B The example is a client proxy mode; in order to distinguish and describe different proxy mode architectures, the first database middleware and the second database middleware are used to distinguish and describe the database middleware under different proxy mode architectures.

[0044] Reference Figure 1A As shown, in some embodiments, under the system architecture 100a corresponding to the server proxy mode, the storage service end 120 includes a first database middleware 121. The first database middleware 121 is a middleware located between the underlying database and the upper-layer application and used to shield the underlying details of each database (for example, a heterogeneous database) in the distributed database. The first database middleware 121 can serve as a bridge for communication between each application 111 in the storage demand end 110 and the distributed database.

[0045] The distributed database includes multiple storage nodes, for example, database A1, database A2 and database A3 are used as examples.

[0046] Reference Figure 1B As shown, in other embodiments, under the system architecture 100b corresponding to the client proxy mode, the storage demand end 110 includes a second database middleware 112. The second database middleware 112 is located between the underlying database and the upper-layer application and is used to shield the underlying details of each database (for example, a heterogeneous database) in the distributed database. The second database middleware 112 can serve as a bridge for communication between each application 111 in the storage demand end 110 and the distributed database. Figure 1BAs shown, in the system architecture 100b corresponding to the client proxy mode, the second database middleware 112 includes multiple connection pools for respectively interacting with the databases to which they are connected. Figure 1B In the example, connection pools 1 to 3 are included. Connection pool 1 is used to connect to and interact with database A1 in the distributed database, connection pool 2 is used to connect to and interact with database A2 in the distributed database, and connection pool 3 is used to connect to and interact with database A3 in the distributed database.

[0047] The data storage method provided in the embodiment of the present disclosure can be executed by the storage service end 120 including the first database middleware 121 in the system architecture 100a, or by the storage demand end 110 including the second database middleware 112 in the system architecture 100b. Correspondingly, the data access method provided in the embodiment of the present disclosure can be executed by the storage service end 120 including the first database middleware 121 in the system architecture 100a, or by the storage demand end 110 including the second database middleware 112 in the system architecture 100b. The following is a detailed description in conjunction with specific embodiments.

[0048] The first exemplary embodiment of the present disclosure provides a method for storing data. The method of this embodiment can be applied to Figure 1A The system architecture 100a includes a storage service end 120 of the first database middleware 121; or applied to Figure 1B The system architecture 100 b includes a storage demand end 110 of a second database middleware 112 .

[0049] Figure 2 The flowchart schematically shows a method for storing data according to an embodiment of the present disclosure.

[0050] Reference Figure 2 As shown, the data storage method provided by the embodiment of the present disclosure includes the following steps: S210, S220, S230 and S240.

[0051] In step S210, a data storage request is received, where the data storage request carries a requesting user identifier and data to be stored.

[0052] In one implementation scenario, refer to Figure 1BAs shown, taking the application as a shopping application as an example, the user (for example, an individual user, a merchant user, etc.) generates business data by operating the application 111. According to the business logic, some business data will be automatically saved to the database (for example, the user's browsing history in the shopping application) or a data storage request will be initiated to the database under the user's save instruction. Correspondingly, the storage demand end 110 receives the above data storage request. The second database middleware 112 of the storage demand end 110 executes the subsequent steps S220 to S240 accordingly.

[0053] In some other implementation scenarios, refer to Figure 1A As shown, the storage demand end 110 forwards the data storage request of the application 111 to the storage service end 120. Accordingly, the storage service end 120 receives the data storage request, and the first database middleware 121 of the storage service end 120 executes the subsequent steps S220-S240.

[0054] In step S220, the corresponding target user data size label is determined according to the requesting user identifier.

[0055] In some embodiments, a user data scale label corresponding to each user identifier is generated in advance in the execution subject (e.g., the storage demand end 110 or the storage service end 120) based on at least one of the historical stored data of different users and the database access performance. For example, the user data scale label of each user can be stored in a user label configuration table.

[0056] For example, the above-mentioned data storage method also includes the step of pre-generating a user data scale label, which specifically includes the following execution steps: generating scale label information within the above-mentioned statistical period for the above-mentioned target user based on the stored data and database access performance of the target user within the statistical period; generating or updating the user data scale label in the above-mentioned user label configuration table based on the above-mentioned scale label information.

[0057] For example, the classification labels of user data scales corresponding to different users can be determined based on at least one of the existing stored data and the current database access performance. When the business data of a certain user exceeds a set threshold (for example, 100 million) or exceeds a certain number level, a large-scale label is assigned to the user. When the business data of a certain user is less than the set threshold or is within a certain data level, a small-scale label is assigned to the user. Alternatively, when the occupancy of the database server disk space exceeds a set ratio (for example, 85% to 90%) or the database access performance jitters, the cause of data access tilt can be located through performance monitoring and diagnosis (for example, through business-level indicator monitoring or database-level performance monitoring, etc.), and the specific user identifier causing the data tilt is determined, and the specific user identifier is assigned a large-scale label. It can be understood that large-scale labels and small-scale labels are used as examples here. In fact, labels of different scales can be divided into more levels, and user labels of 2 levels or more than 2 levels correspond to different packet routing algorithms.

[0058] In other embodiments, considering that the user's data volume changes dynamically over time, the corresponding user data scale label will also change accordingly. Therefore, a label time field is also set in the user label configuration table. The label time is used to indicate the effective time of the user data scale label.

[0059] In step S230, a target group routing algorithm corresponding to the target user data scale label is determined according to the data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data level load between each storage node is balanced.

[0060] In some embodiments, the data routing group configuration information includes: a correspondence between a user data size label and a group routing algorithm. The group routing algorithm and the correspondence may be pre-set by system designers, operation and maintenance personnel, etc. of the data storage service and may support modification and updating.

[0061] The step of determining the target group routing algorithm corresponding to the target user data scale label according to the data routing group configuration information includes: determining the target group routing algorithm corresponding to the target user data scale label according to the corresponding relationship.

[0062] In some embodiments, when the target user data scale label is a user label of the first level scale, the target grouping routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored, so that the data to be stored is divided into a first number of data tables.

[0063] In some embodiments, when the target user data scale label is a user label of the second level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored to obtain a first logical grouping result; performing a second routing grouping operation on the first logical grouping result, so that the data to be stored is divided into a second number of data tables; wherein the second level scale is larger than the first level scale, and the second number is larger than the first number.

[0064] The above attribute information may cover various description dimensions of the data to be stored, including but not limited to one of the following attributes: a requesting user identifier of the data to be stored, a generation timestamp of the data to be stored, a size of the storage space occupied by the data to be stored, etc.

[0065] In some embodiments, the first routing grouping operation may be a hash modulus operation or other operations capable of relatively uniform grouping. The second routing grouping operation may be a grouping operation performed according to a preset time period or a grouping operation in a different operation dimension from the first routing grouping operation.

[0066] In some embodiments, the meaning of data level load balancing between storage nodes can be: the data to be stored is divided into M groups, and these M groups of data are used to store in the corresponding data tables of K storage nodes respectively. The order of magnitude of the stored data in each storage node is equal within the required accuracy, and M and K are both integers ≥2.

[0067] In step S240, according to the target group routing algorithm, the data to be stored are grouped and forwarded to corresponding storage nodes, and a storage mapping relationship between the data to be stored and the stored data table is constructed.

[0068] Based on the target group routing algorithm, the data to be stored can be divided into multiple groups, and the data of each group corresponds to the corresponding data table of the corresponding storage node.

[0069] For example, user identification P1 is identified as a small user tag in the time period T1 to T2, and the corresponding data volume X1 to be stored is 3G (the specific value and magnitude are only for example), among which 0~1G (left open and closed interval, the subsequent representation methods in this paragraph have the same meaning) of data are divided into data table B11 of database A1, 1G~2G of data are divided into data table B21 of database A2, and 2G~3G of data are divided into data table B31 of database A3.

[0070] A hash modulo operation is performed on the attribute information of the data to be stored (an example of the first routing grouping operation), including: based on the hash function, the above attribute information is processed to obtain an output value of a fixed length; then the output value (as the dividend) is subjected to a modulo operation with a target value (as the divisor) (when the dividend and the divisor are both positive numbers, the modulo operation is equivalent to the remainder operation), and the obtained remainder corresponds to the group number to be divided, the target value adopts the total number of groups to be divided (for example, the first number), and the group number to be divided has a corresponding relationship with the data table in each storage node.

[0071] For example, for the data to be stored with small-scale tags, assuming that the first number is 3, by performing the first routing grouping operation, the data to be stored is divided into 3 groups, each group of data has a similar data level, and corresponds to 3 storage nodes (for example Figure 1A and Figure 1B In one storage node in the example databases A1 to A3), each storage node uses a corresponding data table to store grouped data.

[0072] For the data to be stored with large-scale tags, assuming that the second number is 12, the data to be stored is divided into 3 groups by performing the first routing grouping operation, and 3 logical grouping results are obtained; then the data is divided based on the time period in which each group of data is located (an example of the second routing grouping operation), for example, the data of the same quarter is divided into one group, and each group of the above 3 logical grouping results is further divided into 4 groups, and a total of 12 groups are obtained. The data of these 12 groups are stored in 3 storage nodes respectively (for example Figure 1A and Figure 1B In the example databases A1 to A3), each storage node uses four corresponding data tables to store grouped data.

[0073] In some embodiments, the storage mapping relationship is used to represent the mapping relationship between the index information of the data to be stored and the stored data table, and the storage mapping relationship is used to locate the data storage location during the data access process.

[0074] In some embodiments, the index information includes: a user identifier and a user data size tag.

[0075] In some embodiments, considering that the user's data volume changes dynamically over time, the corresponding user data scale label will also change accordingly, so a label time field is also set in the user label configuration table, and the label time is used to indicate the effective time of the user data scale label. The index information includes: user identification, user data scale label and label time.

[0076] For example, in some implementation scenarios, since the actual data volume of the user identifier P1 changes over time, the corresponding user data scale label will also change accordingly; assuming that the user identifier P1 is identified as a large user label after time T2, the corresponding data volume of the data X2 to be stored is 30G, which is divided into 30 data tables through two-dimensional group routing operations. For example, 10 groups of data of 0~1G, 1G~2G, 2G~3G, 3G~4G, 4G~5G, 5G~6G, 6G~7G, 7G~8G, 8G~9G, and 9G~10G are respectively divided into data tables B101, B102, B103, B104, B105, B106, B107, B108, B109, and B110 of database A1; 10G~11G, 11G~12G, 12G~13 The 10 groups of data of 13G to 14G, 14G to 15G, 15G to 16G, 16G to 17G, 17G to 18G, 18G to 19G, and 19G to 20G are respectively divided into data tables B201, B202, B203, B204, B205, B206, B207, B208, B209, and B210 of database A2; 20G to 21G, 21G The 10 groups of data of 2G~22G, 22G~23G, 23G~24G, 24G~25G, 25G~26G, 26G~27G, 27G~28G, 28G~29G, and 29G~30G are respectively divided into data tables B301, B302, B303, B304, B305, B306, B307, B308, B309, and B310 of database A3.

[0077] It is understandable that the above embodiment takes the storage capacity of each data table as an example. In fact, the data volume of each data table can be different, as long as the data volume distribution between each storage node is uniform. Compared with the uniform data volume distribution, the uniform data volume distribution has a looser restriction, allowing slight differences in the data volume between each storage node, and also helps to balance the load between each storage node when accessing data.

[0078] In the embodiment including the above steps S210 to S240, since different users correspond to their own user data scale labels, different group routing algorithms are configured for different user data scale labels in the data routing group configuration information. After receiving a data storage request, the corresponding target user data scale label is determined according to the requesting user identifier, and the target group routing algorithm corresponding to the above target user data scale label is determined according to the data routing group configuration information; accordingly, the corresponding target group routing algorithm is executed, so that user data of different scales can achieve data volume load balancing between each storage node in the process of dividing the data table corresponding to each storage node of the database. On the one hand, uniform group storage under different user data volume levels is achieved, avoiding large differences in the quantity level of data storage between different storage nodes, and effectively solving the problem of data skew; on the other hand, when the user's data volume changes dynamically, the corresponding user data scale label and the corresponding group routing algorithm will also be reflected in the change, so as to dynamically adapt to the situation where the user data volume changes and perform corresponding balanced storage, thereby realizing dynamic adaptive storage.

[0079] Figure 3 The detailed implementation flow chart of step S220 according to an embodiment of the present disclosure is schematically shown.

[0080] Reference Figure 3 As shown, according to an embodiment of the present disclosure, in the above step S220, according to the above requesting user identifier, determining the corresponding target user data scale label includes the following steps: S310 and S320.

[0081] In step S310, based on the requesting user identifier, a match is performed in a pre-configured user tag configuration table to obtain a matching user data scale tag; the user tag configuration table includes: user identifier, user data scale tag and tag time; the tag time is used to indicate the effective time of the user data scale tag.

[0082] The above-mentioned label time can be expressed in various forms. For example, in some embodiments, the label time is expressed by the starting time (also understood as the change time) of the user data scale label, which is used to indicate that the period from the previous starting time to the next starting time is valid. In other embodiments, the above-mentioned label time is expressed by the duration information of the user data scale label.

[0083] For example, the user tag configuration table is as shown in Table 1 below:

[0084]

[0085] In the above Table 1, label 0 is used to represent small-scale user data, and label 1 is used to represent large-scale user data.

[0086] The user tag configuration table also needs to be dynamically maintained as time changes. For example, if the data scale of a certain user changes, adaptability changes or new user data scale tags are required.

[0087] For example, the updated user tag configuration table is as shown in Table 2 below:

[0088]

[0089] Referring to Table 2, it can be seen that for the user with user ID abc5004, from August 1, 2022 to May 1, 2023, the user data scale label of user abc5004 is a small-scale label (the corresponding label value is 0), and from May 1, 2023, the user data scale label of user abc5004 changes to a large-scale label (the corresponding label value is 1).

[0090] In step S320, the target user data size tag corresponding to the reception time of the data storage request is determined according to the matching tag time corresponding to the matching user data size tag.

[0091] Figure 4 The detailed implementation flow chart of step S320 according to an embodiment of the present disclosure is schematically shown.

[0092] In some embodiments, reference Figure 4 As shown, the above label time is represented by the starting time of the user data scale label, which is used to indicate that the period from the previous starting time to the next starting time is valid. In this representation mode, only one transition time and the corresponding user data scale label after the transition need to be recorded in the user label configuration table, which can save the storage space of the user label configuration table and simplify the field storage form.

[0093] In the above step S320, according to the matching tag time corresponding to the above matching user data size tag, determining the target user data size tag corresponding to the reception time of the above data storage request includes the following steps: S410a or S410b.

[0094] In step S410a, when the matching tag time includes multiple matching start times and all of them are earlier than the receiving time, the matching user data scale tag corresponding to the latest matching start time among the multiple matching start times is determined as the target user data scale tag.

[0095] Reference Figure 4As shown, a requesting user identifier corresponds to multiple matching user data scale labels, and the multiple matching user data scale labels have corresponding matching label times, such as the matching start time t11 and the matching start time t12 in the example, t12 is later than t11, and from the matching start time t11 to before the matching start time t12, the label value of the matching user data scale label is 0, that is, a small user label; from the matching start time t12, the label value changes to 1, that is, the matching user data scale label changes to a large user label. In the case where multiple matching start times are earlier than the receiving time t0, the matching user data scale label corresponding to the latest matching start time t12 among the multiple matching start times: a large user label is determined as the target user data scale label corresponding to the receiving time t0.

[0096] In step S410b, when the matching tag time includes multiple matching start times and the receiving time falls within the period between the first matching start time and the second matching start time, the matching user data scale tag corresponding to the first matching start time is determined as the target user data scale tag.

[0097] Reference Figure 4 As shown, one request user identifier corresponds to multiple matching user data scale tags, and the multiple matching user data scale tags have corresponding matching tag times, such as the matching start time t11, the matching start time t12, and the matching start time t13, t12 is later than t11, and t13 is later than t12. From the matching start time t11 to before the matching start time t12, the tag value of the matching user data scale tag is 0, that is, a small user tag; from the matching start time t12 to before the matching start time t13, the tag value of the matching user data scale tag changes to 1, that is, the matching user data scale tag changes to a large user tag; after the matching start time t13, the tag value of the matching user data scale tag changes to 0 again, that is, it changes to a small user tag.

[0098] For the case where the above-mentioned receiving time t0 falls within the time period between t12 (as an example of the first matching starting time) and t13 (as an example of the second matching starting time), the matching user data scale label (for example, a large user label) corresponding to the above-mentioned first matching starting time t12 is determined as the target user data scale label corresponding to the receiving time t0.

[0099] In other embodiments, the above-mentioned label time is represented by the duration information of the user data scale label. In the representation method of this embodiment, since it is difficult to predict the change of the user scale, the current time is used as the end time of the label within any valid time. For example, refer to the illustration in Table 3: when there is no label change, the corresponding duration information is expressed in the form of: starting time-current time. As the current time passes, the end time of the duration information changes dynamically. When the user data scale label changes, the end time of the duration is updated to the final change time. Therefore, the duration information here is constantly changing and updating with the passage of time.

[0100] For example, using this representation method, the example for the situation in Table 2 is as follows:

[0101]

[0102] Among them, in the above-mentioned step S320, according to the matching tag time corresponding to the above-mentioned matching user data scale tag, the target user data scale tag corresponding to the receiving time of the above-mentioned data storage request is determined, including: when the above-mentioned matching tag time contains one or more matching duration period information, determining the target matching duration period information into which the above-mentioned receiving time falls; and determining the matching user data scale tag corresponding to the above-mentioned target matching duration period information as the above-mentioned target user data scale tag.

[0103] Figure 5 The detailed implementation flow chart of step S240 according to an embodiment of the present disclosure is schematically shown.

[0104] Reference Figure 5 As shown, in some embodiments, in the above step S240, according to the above target group routing algorithm, the above data to be stored is grouped and forwarded to the corresponding storage node, and a storage mapping relationship between the above data to be stored and the stored data table is constructed, including the following steps: S510, S520 and S530.

[0105] In step S510, the target group routing algorithm is executed on the attribute information of the data to be stored, and the data to be stored are grouped and divided into corresponding data tables of different storage nodes of the database.

[0106] Reference Figure 1BAs shown, when the execution subject is the storage demand end 110, the second database middleware 112 divides the data to be stored corresponding to different user data scale labels into groups to different degrees by executing the target group routing algorithm, and each group of data corresponds to the corresponding data table of the corresponding storage node. For example, for the data to be stored in the time period T1 to T2 of the user identifier P1, the storage demand end 110 realizes the division of each group of data and the corresponding data table. Afterwards, the storage demand end 110 can send each group of data to the corresponding database A1 to A3 through each connection pool 1 to 3 of the second database middleware 112, for example, through connection pool 1, send 0 to 1G (left open and closed interval, the subsequent expression means have the same meaning in this paragraph) of the data to be stored to database A1, and request to store it in data table B11; send 1G to 2G of the data to be stored to database A2 through connection pool 2, and request to store it in data table B21; send 2G to 3G of the data to be stored to database A3 through connection pool 3, and request to store it in data table B31.

[0107] Reference Figure 1A As shown, when the execution subject is the storage service end 120, the first database middleware 121 divides the data to be stored corresponding to different user data scale labels into groups to different degrees by executing the target group routing algorithm, and each group of data corresponds to the corresponding data table of the corresponding storage node. For the data to be stored of the user identifier P1 in the time period T1 to T2, the storage service end 120 realizes the division of each group of data and the corresponding data table. Afterwards, the storage service end 120 can send each group of data to the corresponding database A1 to A3 through the first database middleware 121. The corresponding data is stored in the corresponding data tables B11 to B31 of each database A1 to A3.

[0108] In step S520, an association relationship between the data to be stored and the stored data table is constructed.

[0109] For example, the constructed association relationship is: {data to be stored 0~1G data → (arrows are used here to represent the association relationship) data table B11 of database A1, data to be stored 1G~2G data → data table B21 of database A2, data to be stored 2G~3G data → data table B31 of database A3}.

[0110] In step S530, the above association relationship is extended according to the requesting user identifier, the target user data scale label and the corresponding label time of the above data to be stored, and a multi-dimensional storage mapping relationship is obtained; the above storage mapping relationship includes: the association relationship between the user identifier, the user data scale label, the label time and the stored data table. The above label time is used to indicate the effective time of the user data scale label.

[0111] After the association expansion, the storage mapping relationship obtained is: {P1→label 0→label time (valid time of label 0)→data table B11 of database A1, P1→label 0 (valid time of label 0)→label time→data table B21 of database A2, P1→label 0→label time (valid time of label 0)→data table B31 of database A3}.

[0112] The above storage mapping relationship is used to represent the mapping relationship between the index information of the data to be stored and the stored data table. The above storage mapping relationship is used to locate the data storage location during data access (broadly speaking, access means, including data query, data modification, data deletion and other operations). This embodiment takes the index information including: user identification, user data scale label and label time as an example. In this embodiment, due to the parameter of the dimension of label time, the scheme can accurately locate the stored data of each historical period, support the setting of filtering period for data query, and for the situation where the routing logic of the stored data table is different due to the change of data volume for the same user, by constructing a storage mapping relationship including label time, in the process of data access, it can match the label time in the index information with the filtering period set during access, so as to locate the corresponding target data table under the group routing algorithm of the filtering period.

[0113] In the solution of storage mapping relationship without label period, it can adapt to the situation where the volume of user data remains unchanged and realize the access of stored data of each user.

[0114] Based on the same concept, the second exemplary embodiment of the present disclosure provides a method for data access, which is applied to a storage demand end including a database middleware. Figure 1B The exemplary system architecture 100b is combined with the solution in which the execution subject in the first embodiment is the storage demand end 110 including the second database middleware 112 to understand the content of this embodiment.

[0115] Figure 6 A flowchart of a method for data access applied to a storage demand side according to an embodiment of the present disclosure is schematically shown.

[0116] Combination Figure 1B and Figure 6 As shown, the method for data access applied to the storage demand side provided by the embodiment of the present disclosure includes the following steps: S610, S620, S630 and S640.

[0117] In step S610, a data access request is received, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed.

[0118] The above data operation instructions include at least one of the following: query instructions, modification instructions, and deletion instructions.

[0119] In step S620, the corresponding access user data scale label is determined according to the access user identifier.

[0120] The specific implementation process of this embodiment can refer to the detailed description of step S220 in the first embodiment mentioned above. The entire implementation process of this step can be incorporated into this embodiment. The only difference is that one is a data storage process and the other is a data access process. The requesting user identifier in the data storage process is replaced by the accessing user identifier in the data access process, and the result is replaced by the accessing user data scale label.

[0121] In step S630, according to the pre-built storage mapping relationship, the target data table corresponding to the access user identifier and the access user data scale label is queried.

[0122] In some embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, and a stored data table.

[0123] In other embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, a tag time and a stored data table, wherein the tag time is used to indicate a valid time of the user data size tag.

[0124] The data access request also carries: a search period for the data to be accessed. In the above step S630, according to the pre-built storage mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale label includes: selecting the target label time matching the search period in the label time of the mapping storage relationship; querying the target data table corresponding to the access user identifier and the access user data scale label within the limited period of the target label time.

[0125] The search period may be used to search for data of one or more time periods, or may be used to search for data of one or more time points.

[0126] In step S640, the data operation instruction for the data to be accessed in the target data table is forwarded to the corresponding storage node.

[0127] Reference Figure 1B As shown, the above data access method may further include: receiving data operation results fed back by each storage node.

[0128] The data access method provided in this embodiment realizes load balancing of the stored data volume among various distributed storage nodes, thereby reducing the occurrence of data skew in the distributed database and effectively avoiding the situation where data access hot spots are concentrated on a certain storage node. In the process of data access, not only the adverse impact of hot spot access on database stability caused by concentration on a certain storage node is avoided, but also the corresponding user data scale label and the corresponding group routing algorithm will be reflected in the case of dynamic changes in the user's data volume, thereby dynamically adapting to the situation of changes in the user's data volume and performing corresponding balanced storage and access, thereby realizing dynamic adaptive storage and access.

[0129] The third exemplary embodiment of the present disclosure provides a method for data access. The method is applied to a storage service end including a database middleware. Figure 1A The exemplary system architecture 100a is combined with the solution in which the execution subject in the first embodiment is the storage service end 120 including the first database middleware 121 to understand the content of this embodiment. The difference between this embodiment and the second embodiment lies in the difference in the execution subject, and accordingly, there is a difference between step S640 and step S740.

[0130] Figure 7 A flow chart of a method for data access applied to a storage service client according to an embodiment of the present disclosure is schematically shown.

[0131] Reference Figure 7 As shown, the method for data access applied to the storage service end provided by the embodiment of the present disclosure includes the following steps: S710, S720, S730 and S740.

[0132] In step S710, a data access request is received, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed. The data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0133] In step S720, the corresponding access user data scale label is determined according to the access user identifier.

[0134] The specific implementation process of this embodiment can refer to the detailed description of step S220 in the first embodiment mentioned above. The entire implementation process of this step can be incorporated into this embodiment. The only difference is that one is a data storage process and the other is a data access process. The requesting user identifier in the data storage process is replaced by the accessing user identifier in the data access process, and the result is replaced by the accessing user data scale label.

[0135] In step S730, according to the pre-built storage mapping relationship, the target data table corresponding to the access user identifier and the access user data scale label is queried.

[0136] In some embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, and a stored data table.

[0137] In other embodiments, the storage mapping relationship includes: an association relationship between a user identifier, a user data size tag, a tag time and a stored data table, wherein the tag time is used to indicate a valid time of the user data size tag.

[0138] The data access request also carries: a search period for the data to be accessed. In the above step S730, according to the pre-built storage mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale label includes: selecting the target label time matching the search period in the label time of the mapping storage relationship; querying the target data table corresponding to the access user identifier and the access user data scale label within the limited period of the target label time.

[0139] The search period may be used to search for data of one or more time periods, or may be used to search for data of one or more time points.

[0140] In step S740, the data operation instruction is executed on the data to be accessed in the target data table to obtain a data operation result.

[0141] The data access method provided in this embodiment realizes load balancing of the stored data volume among various distributed storage nodes, thereby reducing the occurrence of data skew in the distributed database and effectively avoiding the situation where data access hot spots are concentrated on a certain storage node. In the process of data access, not only the adverse impact of hot spot access on database stability caused by concentration on a certain storage node is avoided, but also the corresponding user data scale label and the corresponding group routing algorithm will be reflected in the case of dynamic changes in the user's data volume, thereby dynamically adapting to the situation of changes in the user's data volume and performing corresponding balanced storage and access, thereby realizing dynamic adaptive storage and access.

[0142] A fourth exemplary embodiment of the present disclosure provides a data storage device, which is a storage demand end including database middleware or a storage service end including database middleware.

[0143] Figure 8 The structure of a data storage device according to an embodiment of the present disclosure is schematically shown.

[0144] Reference Figure 8 As shown, the data storage device 800 provided by the embodiment of the present disclosure includes: a storage request receiving module 810, a storage tag determining module 820, a routing algorithm determining module 830 and a packet routing module 840.

[0145] The storage request receiving module 810 is used to receive a data storage request, where the data storage request carries a requesting user identifier and data to be stored.

[0146] The storage label determination module 820 is used to determine the corresponding target user data size label according to the requesting user identifier.

[0147] The above-mentioned routing algorithm determination module 830 is used to determine the target group routing algorithm corresponding to the above-mentioned target user data scale label according to the data routing group configuration information; the above-mentioned data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data level load between each storage node is balanced.

[0148] The above-mentioned group routing module 840 is used to group the above-mentioned data to be stored and forward it to the corresponding storage node according to the above-mentioned target group routing algorithm, and construct a storage mapping relationship between the above-mentioned data to be stored and the stored data table.

[0149] In some embodiments, the above-mentioned device 800 also includes: a label configuration module.

[0150] The above-mentioned label configuration module is used to: generate scale label information within the above-mentioned statistical period for the above-mentioned target user based on at least one of the stored data and database access performance of the target user within the statistical period; and generate or update the user data scale label in the above-mentioned user label configuration table based on the above-mentioned scale label information.

[0151] For other relevant details or beneficial effects of this embodiment, please refer to the contents of the first embodiment and will not be repeated here.

[0152] A fifth exemplary embodiment of the present disclosure provides a data access device.

[0153] Fig. 9 The structure block diagram of a data access device according to an embodiment of the present disclosure is schematically shown.

[0154] Reference Fig. 9 As shown, the data access device 900 provided in the embodiment of the present disclosure is a storage demand end including a database middleware, including: a first access request receiving module 901, a first access tag determining module 902, a first storage table query module 903 and a forwarding module 904.

[0155] The first access request receiving module 901 is used to receive a data access request, which carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0156] The first access tag determination module 902 is used to determine the corresponding access user data size tag according to the access user identifier.

[0157] The first storage table query module 903 is used to query the target data table corresponding to the access user identifier and the access user data scale label according to the pre-built storage mapping relationship.

[0158] The forwarding module 904 is used to forward the data operation instruction for the data to be accessed in the target data table to the corresponding storage node.

[0159] For other relevant details or beneficial effects of this embodiment, please refer to the contents of the second embodiment, which will not be repeated here.

[0160] A sixth exemplary embodiment of the present disclosure provides a device for accessing data.

[0161] Fig.10 The structure block diagram of a data access device according to another embodiment of the present disclosure is schematically shown.

[0162] Reference Fig.10 As shown, the data access device 1000 provided in the embodiment of the present disclosure is a storage service end including a database middleware, including: a second access request receiving module 1001, a second access tag determining module 1002, a second storage table query module 1003 and a data operation module 1004.

[0163] The second access request receiving module 1001 is used to receive a data access request, which carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction.

[0164] The second access tag determination module 1002 is used to determine the corresponding access user data scale tag according to the access user identifier.

[0165] The second storage table query module 1003 is used to query the target data table corresponding to the access user identifier and the access user data scale label according to the pre-built storage mapping relationship.

[0166] The data operation module 1004 is used to execute the data operation instruction on the data to be accessed in the target data table to obtain the data operation result.

[0167] For other relevant details or beneficial effects of this embodiment, please refer to the contents of the third embodiment, which will not be repeated here.

[0168] Any multiple of the functional modules included in the above-mentioned device 800, device 900 or device 1000 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules included in the device 800, device 900 or device 1000 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the functional modules included in the device 800, device 900 or device 1000 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0169] A seventh exemplary embodiment of the present disclosure provides an electronic device.

[0170] Fig.11 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown.

[0171] Reference Fig.11 As shown, the electronic device 1100 provided by the embodiment of the present disclosure includes a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102 and the memory 1103 communicate with each other through the communication bus 1104; the memory 1103 is used to store computer programs; the processor 1101 is used to implement the data storage method or data access method as described above when executing the program stored in the memory.

[0172] The eighth exemplary embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the data storage method or data access method described above is implemented.

[0173] The computer-readable storage medium may be included in the device or apparatus described in the above embodiment; or it may exist independently without being assembled into the device or apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0174] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0175] It should be noted that the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution provided by the embodiments of the present disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0176] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0177] The foregoing is merely a specific embodiment of the present disclosure, which enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for data storage, characterized in that: include: Receive a data storage request, the data storage request carrying a requesting user identifier and data to be stored; Determine a corresponding target user data size label according to the requesting user identifier; Determine a target group routing algorithm corresponding to the target user data scale label according to the data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database so that the data level load between the storage nodes is balanced; According to the target group routing algorithm, the data to be stored are grouped and forwarded to corresponding storage nodes, and a storage mapping relationship between the data to be stored and the stored data table is constructed.

2. The method according to claim 1, characterized in that The data routing group configuration information includes: a correspondence between a user data scale label and a group routing algorithm; The step of determining the target group routing algorithm corresponding to the target user data scale label according to the data routing group configuration information comprises: determining the target group routing algorithm corresponding to the target user data scale label according to the corresponding relationship; In the case where the target user data scale label is a user label of the first level scale, the target group routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored, so that the data to be stored is divided into a first number of data tables; In the case where the target user data scale label is a user label of the second level scale, the target grouping routing algorithm includes: performing a first routing grouping operation on the attribute information of the data to be stored to obtain a first logical grouping result; performing a second routing grouping operation on the first logical grouping result, so that the data to be stored is divided into a second number of data tables; wherein the second level scale is larger than the first level scale, and the second number is larger than the first number.

3. The method according to claim 2, characterized in that The first routing grouping operation includes: a hash modulus operation; The second routing grouping operation includes: performing grouping operation according to a preset time period; The attribute information includes: requesting user identification.

4. The method according to claim 1, characterized in that: Determining a corresponding target user data size label according to the requesting user identifier includes: Based on the requesting user identifier, a match is performed in a pre-configured user tag configuration table to obtain a matching user data scale tag; the user tag configuration table includes: a user identifier, a user data scale tag and a tag time; the tag time is used to indicate the effective time of the user data scale tag; According to the matching tag time corresponding to the matching user data size tag, a target user data size tag corresponding to the receiving moment of the data storage request is determined.

5. The method according to claim 4, characterized in that The label time is represented by the starting time of the user data scale label, and is used to indicate that the period from the previous starting time to the next starting time is valid; Wherein, determining the target user data scale tag corresponding to the reception time of the data storage request according to the matching tag time corresponding to the matching user data scale tag includes: In a case where the matching tag time includes multiple matching start times and all of them are earlier than the receiving time, determining the matching user data scale tag corresponding to the latest matching start time among the multiple matching start times as the target user data scale tag; When the matching tag time includes multiple matching start times and the receiving time falls within a period between a first matching start time and a second matching start time, the matching user data scale tag corresponding to the first matching start time is determined as the target user data scale tag.

6. The method according to claim 4, characterized in that The tag time is represented by the duration information of the user data scale tag; Wherein, determining the target user data scale tag corresponding to the reception time of the data storage request according to the matching tag time corresponding to the matching user data scale tag includes: When the matching tag time includes one or more matching duration information, the target matching duration information into which the receiving moment falls is determined; and the matching user data scale tag corresponding to the target matching duration information is determined as the target user data scale tag.

7. The method according to claim 4, characterized in that Also includes: Generate scale label information within the statistical period for the target user according to at least one of the stored data and database access performance of the target user within the statistical period; According to the scale label information, the user data scale label in the user label configuration table is generated or updated.

8. The method according to any one of claims 1 to 7, characterized in that The method is applied to a storage service end including a database middleware or a storage demand end including a database middleware; According to the target group routing algorithm, the data to be stored are grouped and forwarded to corresponding storage nodes, and a storage mapping relationship between the data to be stored and the stored data table is constructed, including: Executing the target group routing algorithm on the attribute information of the data to be stored, and dividing the data to be stored into corresponding data tables of different storage nodes of the database; Constructing an association relationship between the data to be stored and the stored data table; According to the requesting user identifier, target user data scale label and corresponding label time of the data to be stored, the association relationship is associated and expanded to obtain a multi-dimensional storage mapping relationship; the storage mapping relationship includes: the association relationship between the user identifier, the user data scale label, the label time and the stored data table.

9. A method for data access, characterized in that: Applied to a storage demand side including database middleware, the method comprises: Receive a data access request, the data access request carrying: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; Determine a corresponding access user data scale label according to the access user identifier; According to the pre-built storage mapping relationship, query the target data table corresponding to the access user identifier and the access user data scale label; The data operation instruction for the data to be accessed in the target data table is forwarded to the corresponding storage node.

10. A method for data access, characterized in that: Applied to a storage service client including database middleware, the method comprises: Receive a data access request, the data access request carrying: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; Determine a corresponding access user data scale label according to the access user identifier; According to the pre-built storage mapping relationship, query the target data table corresponding to the access user identifier and the access user data scale label; The data operation instruction is executed on the data to be accessed in the target data table to obtain a data operation result.

11. The method according to claim 9 or 10, characterized in that: The data access request also carries: a search period for the data to be accessed; The storage mapping relationship includes: the association relationship between the user identifier, the user data scale label, the label time and the stored data table; the label time is used to indicate the effective time of the user data scale label; According to the pre-built storage mapping relationship, querying the target data table corresponding to the access user identifier and the access user data scale tag includes: Filtering the target tag time matching the search period from the tag time of the mapping storage relationship; Within the limited time period of the target tag time, query the target data table corresponding to the access user identifier and the access user data scale tag.

12. A data storage device, characterized in that: The device is a storage demand end including a database middleware or a storage service end including a database middleware, and the device includes: A storage request receiving module, used to receive a data storage request, wherein the data storage request carries a requesting user identifier and data to be stored; A storage label determination module, used to determine a corresponding target user data size label according to the requesting user identifier; A routing algorithm determination module, used to determine a target group routing algorithm corresponding to the target user data scale label according to data routing group configuration information; the data routing group configuration information is used to divide user data of different scales into data tables of different storage nodes of the database, so that the data level load between each storage node is balanced; The grouping routing module is used to group the data to be stored and forward them to corresponding storage nodes according to the target grouping routing algorithm, and to construct a storage mapping relationship between the data to be stored and the stored data table.

13. A data access device, characterized in that: The device is a storage demand end including database middleware, and the device includes: A first access request receiving module, configured to receive a data access request, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; A first access label determination module, used to determine a corresponding access user data scale label according to the access user identifier; A first storage table query module, used to query a target data table corresponding to the access user identifier and the access user data scale label according to a pre-built storage mapping relationship; The forwarding module is used to forward the data operation instruction for the data to be accessed in the target data table to the corresponding storage node.

14. A data access device, characterized in that: The device is a storage service end including a database middleware, and the device includes: A second access request receiving module is used to receive a data access request, wherein the data access request carries: an access user identifier and a data operation instruction for the data to be accessed; the data operation instruction includes at least one of the following: a query instruction, a modification instruction, and a deletion instruction; A second access label determination module, used to determine a corresponding access user data scale label according to the access user identifier; A second storage table query module, used to query the target data table corresponding to the access user identifier and the access user data scale label according to the pre-built storage mapping relationship; The data operation module is used to execute the data operation instruction on the data to be accessed in the target data table to obtain the data operation result.

15. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 11 when executing a program stored in a memory.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images